Twitter BFFs

Mentions

So if you wanted to make a very basic BFF app, you would probably take the means of the numbers of mentions between a user and each of their followers, then pick out, say, the 5 highest.

So maybe define R(a,b) for two users as the number of times 'Alice' mentions 'Bob', and R(b,a) vice versa (assuming that both counts are taken over the same period of time).

In this instance, I would probably go with the geometric mean,

since this would weed out any one-sided relationships by making the result zero.

You'd probably also want to factor in whether they're following each other, so define

and multiply the mean by it. Or, in other words, make it zero if they don't both follow each other.

And that would suffice.

The only major downside to this is - imagine a pair of people who talk a massive amount, but all of what they say to each other is abusive. BFF wouldn't be the best way to describe them.

Unfortunately, unless you add "sentiment analysis" to the app - which is a little hit and miss at present - you're just going to have to assume everyone talks nice to each other.


My BFFs

Now, this is another situation where the best numbers aren't readily available - those ideally being:
1) date started following
2) total number of @replies since then

This obviously raises the same problems I talked about in the previous post. But for demonstrative purposes, I'll use the number I can get.

So for this, I'm using Twoolr, over the period Sept 1 - Oct 18 (48 days).

Now a problem arises here. Twoolr seems to be missing a significant number of people for the second column, who I know for a fact have @replied me during that period. I don't know what's going on there, and unfortunately there's not much I can do to help it.

So the last column isn't really much use. But it could suffice to take, say, the top 5 and call them my Twitter BFFs over that 48 day period, if only so we have something vaguely resembling an answer.

Or alternatively I can just go here, click on 'closest friends', and get this:

Covering about 2 week. But then, that particular site's numbers haven't been entirely reliable in the past..


Accounting for Rates

Given that people tweet at different rates - and may talk a lot with almost all their follows - it would probably make sense to take this into account.

For this I would probably say take

i.e. the proportion of Alice's total @replies that are directed at Bob. So for my numbers above the total is 525, and you'd divide everything through by that.

So then you can work that out for R(a,b) and R(b,a), take the geometric mean as before, multiply by 100 and your result should be a value between 0 and 100 vaguely indicating how 'close' Alice and Bob are.

Which sounds ridiculously straight forward. But that only works if you can get all the numbers. But that would require authorisation from both users in question, which in practice isn't easy.

Incidentally, this is why people are so desperate and willing to buy, sell and steal your personal data. It's valuable! Especially to marketing people.


Improving the Results

So there are other things you might want to include in the calculation to improve the 'quality' of the results. That is, there are other things we can measure that are also good indications of friendship and closeness.

Again, the absolute ideal way of working out how good friends people are would be to go through everything they send to each other (and not just on Twitter) to look at what exactly they're saying. But that's time consuming, and people tend not to like it when you invade their privacy and analyse what they say, to that great a degree.

1) Network

Basically, how many friends Alice and Bob have in common. You could even go so far as to work out the biggest 'clique' the two are part of - that is, the largest group Alice and Bob are in, such that everyone in that group are friends with each other.

So in general, you'd expect Alice to be closer friends with Bob than Carol if Alice and Bob have more friends in common - or are part of a large clique - than Alice and Carol.

I updated the network map (old version here) to clarify this point, and also just because I can.

[click to embiggen. Interactive version with me here. Version without me here.]

2) Retweets

This tends to be better at showing how interesting a user finds another's tweets. So if you did decide to include it, you probably wouldn't give it much weight.

But at the same time, one would like to believe that - if for example someone you liked and someone you were indifferent to tweeted the same thing - you'd be more likely to retweet the person you liked. So it's mildly worth considering.

3) Follow Friday

Like retweets, this can indicate how interesting a user is as much as it can indicate friendship. Ideally, you'd want to consider the reason (if any) accompanying the #FF tweet.

4) Lists

This is another one that's only useful if you can interpret how the lists are defined. Being on a list called "arseholes" for example, doesn't exactly suggest friendship.


But as I say, if you don't want things to get over-complicated, you can just ignore all the above.


The Friendliest Tweeter

If you want to be really clever, you could work out the numbers for all of a user's followers, then combine them (in some way) to get a 'friendliness score' for that user.

Which is nice, because it would give people a way to compete over who's the friendliest (with their follows/followers).

Possibly more on that in a later post.


And yes, the title was a bit of a cheat since I can't actually tell you who my Twitter BFFs are. Sorry.


Oatzy.

Thoughts on the Feasibility of Pokémon

If I were any good at drawing, this would be illustrated. But I'm not, so it isn't.

First of all, if Pokémon were made real, battling them would pretty much be out of the question, since it would likely be considered animal cruelty (akin to cock fighting, etc). Plus, you'd have lots of Daily Mail readers bitching about how genetic engineering is unnatural or whatever. But for convenience, I'm ignoring these facts.

Second, I should point out that I'm not a biologist, so if I get anything wrong - Mrs Elston, Mr Bestford, I apologise.

All this will assume current or plausible future technology. This is not to say that some of the things I may dismiss will always be impossible. You never know really.


So there are three key things you need to work out if you're making pokémon:
1) aesthetic correctness
2) "evolution"
3) "moves" - whether they're possible and how.


Possibility

First and for most you have to ask, is a particular pokémon possible at all?

And for this I mean like, for example, ghost pokémon - no, ghosts don't exist. Pokémon made of solid stone or steel - no, I can't really see how that would work.

But aside from those, there are some that would be possible if you made minor (or slightly major) changes.

One example - pokémon that are perpetually on fire. Aside from health and safety issues, I can't see how you'd make it work. Especially for Ponyta and Rapidash - a horse with a flaming mane? I can't see it ending well. Unfortunately, if you take away the fire, they're pretty much just horses.

Similarly, things like Blastoise - the giant turtle with the retractable metal water cannons in it's shell - definitely needs some tweaking. Maybe if they were fixed, bone-based cannons that were some how attached to something akin to a blowhole?

With regards steel/stone pokémon, it could work as either (pseudo) sentient robots for the likes of Voltorb, Magnemite, etc; or in the case of say Onix, maybe a large snake with a hard, rock-like armor. And similar for other rock pokémon.

But these tricky ones aside, there are plenty that are quite plausible.


Creation

There are two methods of creation, as it is. In the cases of (mostly) aesthetic variations of pre-existing animals, you could get away with selective breeding, at least to a greater extent.

This would work best in, for example, some bird pokémon like Pidgey, Swellow, Starly; or some reptiles like Ekans, Arbok, Kecleon..

And in some cases, like Eevee - setting aside the whole evolutions thing (more on that later) - you could quite easily have an Eevee by just buying an appropriate breed of dog (pomeranian?) and grooming so as to give it the correct appearance.

Similarly for things like Teddiursa and Ursaring that need little to no change from pre-existing animals.

In terms of markings, some may be harder to breed for definite recurrence - i.e. there'll be a lot of randomness in specific markings. And there are some colours, or fur in particular, which I'm not entirely sure at possible naturally.

For ones that don't significantly resemble existing animals, but resemble extinct animals such a dinosaurs, it'll be a wait. Beyond that, you may struggle.

Things like Bulbasaur, I don't know how you'd graft a living plant onto a reptile - unless of course it's not actually, physically a part of the reptile, but has a symbiotic relationship with it. Somehow.

Anyway. I could go through all the pokémon, but frankly I'm not going to. 600+ pokémon? Pfft!

Maybe some other time.


Evolution

First of all, the pokémon definition of 'evolution' is quite different to Darwinian evolution. It's wrong for one thing. But that's incidental.

The whole evolution thing is a pretty big part of pokémon. But I'm just going to go ahead and say this - dramatic, instantaneous metamorphosis is ridiculous and impossible.

Realistically, it has to happen over an extended period - so for example, with the Pidgey evolutions the change would work gradually, and you'd have intermediary phases between it and Pidgeotto and Pidgeot. And this would work for a lot of pokémon, albeit with a little creativity.

Insect pokémon, at lot of them mimic real insects pretty well anyway. They just speed things up, missing out the gradual change.

Obviously, this evolution would be time-dependent rather than as a result of 'experience'. Which does take the fun out of it slightly, since it would make the evolution of you pokémon feel like less of an achievement.

Plus it would mean that your little sister could have a Gyarados just by catching a crappy little Magikarp and waiting. And besides that, Magikarp's evolution would be less impressive if it were gradual. Unless some sort of cocoon like deal were used.

The more 'magical' means of evolution - i.e. stones, trading; you might be pushing it.

I suppose, in the case of stones, they could trigger specific genes to activate - either by being radioactive, or some such crap - and again, over time, that would lead to a particular final outcome. Trading, I can't see that ever having an effect. Certain held items, pretty much the same deal as the stones.


Battling and Abilities

Okay, so assuming your pokémon doesn't win a battle, sustaining only minor injuries (if any), in the real world there are really two outcomes to a battle:

1) Unable to continue - either by being knocked-out, comatose, crippled - and I mean genuinely crippled. These things could potentially be healed over time - but not as instantaneously as the Pokémon Centre does - and odds are in most cases they'll never be able to fight again.

2) Dead.


Normal, Fighting, Flying, Rock, Steel, Dragon, Ground, Bug,.. - pretty much any physical attack, fair enough. It may take some training or may have to be taught, but still.

Certain attacks may not be possible for certain animals. Whatever. It's straight-forward enough.

Most defensive attacks are pretty straight-forward as well. Falling asleep in the middle of a battle - unlikely. And I doubt the sleep's restorative power would be worth it.

Special attacks. Now:
Psychic - might be vaguely possible in the future. Maybe. But that might be pushing it. Would likely be in the form of some implantable technology.

Electric - most plausible of the specials, given that creatures such as electric eels already do this sort of thing, albeit short range. You might be able to enhance that.

Water - possible, in so far as it's possible to manipulate any water that may be around you. Out of water, you might have a problem, unless they can store it somehow, or you just carry bottles of water with you.

Ice - similar to above. You can manipulate surrounding ice/snow, but I can't think how it'd work away from snow.

Fire - not that sure. Suggestion have been put forward for how fire-breathing dragons could work. Maybe by storing naturally produced gases (methane, for example) and creating some sort of spark using claws or teeth or something as a sort of flint. Hell, hold a lighter in front of a cow's mouth and wait - fire-breathing cow! So yeah, it's mildly possible.

Grass - again, it's manipulating your environment, or possibly using any foliage that might be a part of your body.

Poison - fairly self-explanatory.

Ghost - no. Non-physical Dark - no. Other specials of the more physical attacks - might be pushing it

And again, all of this comes down to some genetics, and some training (as appropriate).

Abilities, some are plausible - such as having good balance, being flame-resistant, etc - while others are slightly less so - like levitating.


Bonus: Shiny Pokémon

These are Pokémon with alternate colouration - a minor aesthetic difference. So realistically, while we're genetically engineering them, we can give then a recessive gene, or else a set of genes prone to mutate in rare instances - that will cause the change in appearance in approx. 1 in 8192 individuals of a species.


Technology

Pokéball - for the foreseeable future, out of the question. Not until (or unless) we develop transportation and similar derivative technologies. I wouldn't hold your breath.

Obviously, this presents a problem - in the cases of the larger pokémon, where do you keep them? In a good handful of cases, there are Pokémon too large to keep even in your back garden.

So either make them such that they don't exceed some size (say, some zoo friendly size) or else try to avoid those which will cause this problem.

Healing - as discussed above, I can never imagine a way of healing potentially fatal injuries in 5 seconds or less. We're talking standard veterinary care - albeit in the future slightly more advanced. And the vets would, first, have to be trained to treat Pokémon.

Potions - I can't think of a real world equivalent, besides general curative pharmaceuticals.

Paralyse heal - only if it's, say, hysterical-paralysis or down to muscles tightening or something of that sort. You certainly can't spray a magic potion and heal broken bones.

Awakening - maybe. Though it most cases, just waiting would be sufficient.

Antidote - i.e. anti-venom, sure, but I don't know if an all-in-one is possible.

Burn Heal - Yeah, but it won't be instantaneous.

Ice Heal - again, just giving it time would suffice, or else wrapping them up for quicker results. Any worse than that, your Pokémon's fucked, unless it can produce it's own anti-freeze.

Full Heal, Full Restore and berries - don't push your luck.


Conclusion

So some pokémon aren't entirely implausible. Which is nice. Others, no chance.

As Joe pointed out, there are risks with genetically engineering pokémon. Lord knows we wouldn't want a Mewtwo on our hands. (Or would we?). Although, given all the rigor that goes into genetic engineering, we shouldn't have such dramatic problems.

But it's worth keeping in mind, there is always the risk of unforeseen and unintended consequences. Ideally, you'd want to make them - at least initially - on a Jurassic Park -type island, where they'd be contained with minimal risk to the general public.

And as a final thought, if we keep in mind that scientists haven't made unicorns yet - which is a fairly minor alteration to a pre-existing animal - we might be waiting a while.


Oatzy.


Bonus - various photoshops of 'real' Pokémon:

Follow Up: The Women Really Do Tweet More!

As Aerliss quite rightly pointed out, taking an average over the last 2 weeks isn't that accurate for anyone who has been uncharacteristically quite or talkative over that period. Problem was, I couldn't get numbers for further back than that.

In fact, it turns out the numbers I would've gotten with the Twitter API, I can get in browser by going to, for example http://twitter.com/statuses/friends/oatz.xml.

And what you can get in there is the date a user joined and their total number of tweets, which means you can work out a 'lifetime average'.

For those interested, here is the code that turns date joined into number of days online.

Anyway. I ran the numbers. Some were surprisingly close to those I got yesterday. Others were wildly out.


The Numbers

What I found was that there are 3 particular outliers - 3 women who are WILDLY more talkative that anyone else I follow.

So for the sake of naming and shaming, and so you can all feel vaguely competitive with each other, here are the numbers:

Sorted by rate, coloured by gender. Click to enlarge.


Needless Complaining

Now, I do still have some qualms with this approach. For example, I joined last May. But there was a period of around 6 months when I never went on Twitter. So if we could exclude that period my average would be more representative of my tweet rate (and higher). That said, my average is still one of the higher ones.

I don't know if and how this affects other people, but there are some cases where the lifetime-averages are dramatically different to the 2week-averages.

Still, it does give me some of the numbers that were missing from yesterday's. So pros and cons. But in all honesty, drawbacks aside, I do trust these numbers more than yesterday's.


The Results

Enough waffling, here are the revised results:

Men:
Average = 5.29 tweets/day
Standard Deviation = 4.93

Women:
Average = 19.21 tweets/day
Standard Deviation = 24.94

So yeah. I wasn't imagining it. The women I follow really do tweet (on average) a lot more than the men!


A Graph

To show the spread - but mostly just because I can - here are the rates plotted on a graph (natural-logarithmic scale). Blue squares are women, orange diamonds are men.

So with the women you have a big clump in the middle and a handful at the extremes, whereas the men are sort of more evenly spread.

Even if you exclude the 3 female 'outliers', the average - 7.28 - is still higher than the men's, although the standard deviation becomes lower than the men's.

If you include me in the men's average, theirs still only goes up to 5.79.

So there you have it.


Random facts

- Of the people I follow, abooth202 has been on Twitter the longest - 966 days - with 5th November 2010 being his 1,000th day.

- miss_popcouture has posted the most tweets in her time on Twitter, with a heroic 49,922.



Oatzy.


* All figures correct as of 2nd October 2010. Subject to variation over time.

More Twitter Insights

Introductions

First of all, following a few recent new follows, here's the revised introductions graph.

[Click to enlarge. Interactive version here.]

Main new thing going on is that some of the chains are getting longer. Also I put in some of the people I missed out last time (for clarity), just so it's more complete.


Gender

Recently, I thought I noticed that a lot of my timeline was filled with tweets by women-folk. Obvious first conclusion being that I must now be following more woman than men.

Having looked into this matter further, I actually found that I'm following almost equal numbers of men (15) and women (14) - excluding celebrities and dead-accounts.

Next question then is - What the hell?

Okay, this next theory may sound a little sexist but bear with me - what if it's just that the stereotypes are true and women really do talk more (on average)?

Given that a recent update to the Twitter API borked my previously mentioned programs and I don't (yet) know how to fix them, I had to get numbers and such by hand. Which made life a little harder.

Individual tweet rates are only approximate, and based on an average over the last ~14days. And I'll be honest, I don't entirely trust the source numbers. Also, I couldn't get numbers for some people, which isn't ideal.

NB/ For more applicable results, it would've made sense to only count tweets that would have shown up in my timeline - i.e. excluding tweets @ people I don't follow. But working that out would require a lot more effort, and frankly I don't care enough about that degree of correctness to bother.

Anyway, here are the results:

Men:
Average = 5.2 tweets/day
Standard Deviation = 4.29

Women:
Average = 6.6 tweets/day
Standard Deviation = 4.81

So yeah. The women tweet about 27% more than men. They also have a slightly greater spread of rates.

Basically, the women that tweet the most, tweet more than the men who tweet the most.  Which pushes the average and spread up.

Again, these results are only approximate and only apply to my personal network, and could vary (significantly) over time. Also if we had the missing data, it could turn out the opposite is actually true.

In fact, someone already did research on this matter for a random sample of 300,000 Twitter users. What they found was that, while there are slightly more women (55%) on twitter than men,

We found that an average man is almost twice more likely to follow another man than a woman. Similarly, an average woman is 25% more likely to follow a man than a woman.

And that, on average, men and women tweet at about the same rate.

Maybe I'm just following particularly talkative women/quiet men...


Location

Just because I can.

These are maps of people who follow me, rather than just people I follow (which I would've prefered). But I couldn't find something that could do that, so gave up and settled on this. Interactive version and make your own here.

World View

UK View

[Click to enlarge.]


Bios

For the people in the graph at the top of the page, I took their bios and made this word cloud [click to enlarge. Interactive here.]:

This is the company I keep - geeks and writers.


And One More Thing

Previously mentioned Malcolm Gladwell wrote an article for The New Yorker recently, about activism in social networks - Twitter in particular - called "Why the Revolution Will Not Be Tweeted".

In it, he explains how and why social-media based 'activism', is quite different from and less effective than real-world activism - the sort of activism that brings about genuine change. Rather, social-media activism is good at getting lots of people to participate, but they (mostly) only do so with the least amount of effort.

So, for example, they might join a FB group, sign an online petition, or even do the sort of crap 4chan pulls. But they tend not to actually go out and protest, where genuine commitment is needed and where there's the risk of say physical harm. And the resulting pay-off is much less significant as a result.

"This is because, Gladwell says, online networks are all about weak ties — a weak tie is a friend of a friend, or a casual acquaintance — whereas real activism depends on strong ties, or those people you know and trust"

In response, Jonah Leher - writing for Wired - argues that Gladwell's dismissal of weak ties in social activism may be a little short sighted. And in particular it can be necessary for a leader of a cause to have lots of weak ties, so as to have greater reach - or at least, this seems to be the case in real world situations.

Gladwell also talks about how a hierarchically structured group is more effective in activism than a decentralized-network structured group - as is often the form online groups take. And again, this is required for greater levels of discipline, control and commitment.

Both make good arguments, and both articles are worth reading.


Oatzy.