I’ve recently started using Rockstar 365 which is all about scoring and ranking the best Microsoft professionals on the planet.
So initially I thought this sounded like quite a cool concept; you make a profile, add some project experience, blog/twitter links, exams and certificates, then go to sleep. You wake up the next day and hopefully you will have a rank respectable enough it doesn’t immediately send you scrambling to find the delete button.
As of writing I am currently #27 in the UK – first thoughts? I’m pretty great. 😛
However this immediately raises a few questions, primarily what does the 27 mean? 27th most experienced? Most valuable? Most skilled? Most certificated?
The number on its own doesn’t say a lot, it doesn’t have any context. For the sake of argument I’m the 27th best Microsoft Professional in the UK, however that doesn’t really mean a lot on its own, the best at what? The list of Microsoft technologies is almost endless, so I’ll focus on CRM (the stuff I do most days). Most people who do CRM, do one thing in particular (PM, technical, consultant, pre sales, etc), but there is no sub ranking by role, the 27 is too broad in that context, it’s meaning is so broad it becomes misleading. E.g. do you hire the developer at #1 even though you actually want a consultant, the highest of which is ranked at #20? You would have to start looking at their experience for more context – but then isn’t that just LinkedIn?
There is a major question around the number of users who have signed up. 27 out of thousands sounds incredible, 27 out of 30? Not so much. This information doesn’t appear to be accessible. This is probably linked to Rockstar’s (very reasonable and understandable) stance explained here. I’m going to guess that the current user base is a small percentage of all Microsoft professionals, but then that means I’m 27 just because most people didn’t sign up, the number can’t be trusted, and if it can’t be trusted, then how can it be used for sensible decision making?
A lack of transparency feels like a serious issue, I don’t know how the 27 is even calculated. Is it just on the information I enter onto profile? In which case is it basically just a calculator for certificates and job experience? Does it take into account more abstract measures such as the number of tweets, followers, blog posts, profile popularity? What about involvement in the MSDN forums? But I use StackOverflow, so am I missing out because of my forum choice? There are just a huge number of unknowns, and if you don’t know how something works how can you trust the result it gives? The Microsoft Expertise Level also suffers from the same issues.
At the moment I feel like I’m in a race; where I have no idea who I’m racing against, which direction I should be running in, what the rules of the race are or what my position actually means.
Now that was all pretty negative, but I feel its fair criticism and I hope its not interpreted as criticism just for the sake of criticism. Rockstar 365 is still in Beta and is a relatively new development and from reading their blog, they are still developing and delivering new ideas – so maybe its unfair to judge at this early stage. In terms of the issues I’ve raised above I believe there is an answer to each one:
- Context – In their blog post I’ve linked above they talk about creating a contextual search which should hopefully make it more obvious what the rank actually means. E.g. the rank is relative to the search being performed.
- User base – It would be nice to get some numbers on registered users (and those users who are active), but this might just be one of those issues that resolves itself (if the service becomes sufficiently popular).
- Transparency – This could probably be resolved by a publication, which explains how everything is calculated, then at least everyone would know why they have the number they have.
I’m not giving up Rockstar 365, I just don’t get it, yet.