Posts Tagged Analytics
Why are they broken? Because I like them, because I like charts, graphs and numbers, and because the tools are awesome. Ok, that doesn’t really explain why I think analytics are broken, but it might help explain why I feel this way. So if the tools are so awesome and easy to use, why are analytics not use properly? Because it is the actual process that is broken (well, I’m talking for the rich web experience world because that is what I know).
How it should be
I guess people think implementing analytics is easy so they don’t give it enough time. The part that is not working currently is not the implementation but the analysis. And the analysis part is the most important, if you don’t do it, there is nearly no point to implementing analytics. In my head you could divide the analysis part in two, first would be building an hypothesis document about what the final data is going to be (ex: we are going to try this interface and it is going to have this impact on data), and secondly an analysis document that review the previous hypothesis, tries to explain why things went well or not and what to do the next time to improve. The thing is, none or only part of that is happening right now. Having made these steps will also make implementation more coherent (and no, just tracking everything and not caring is not a solution).
The problem: finding the right person
I think that the whole problem comes from the fact that it is not the right person in charge of the analytics. Most of the time one of two things happens. Either the person in charge of analytics is a so-called analytics expert paid by the client (so separated entity than the production company), that is never talked to, that just provides an excel sheet with stuff to track, he usually works way before the project is completed (so project could change a lot meanwhile), and once this is done, he is never heard of again. Most of the time the goal is to integrate the data of this campaign into the data of the main site. The problem is that campaigns (rich experiences) needs to be tracked differently so the data probably integrates badly anyway. The second case I have seen is where the developer is the one in charge of analytics. I think this approach is already a bit better because you can’t get someone closer to the project than the developer. The problem is that most of the time, he doesn’t have access to the data once the project is completed, so no post analysis can be done or at least not by the person who would know what to analyze.
Who is that right person?
For me the right person would either be an information architect or a usability expert. These people are close enough to the project plus they are the one who need to check if they what they did was right using the real numbers. These guys could write proper hypothesis before the project, write thoughtful analysis and they can point out what to improve next time. Actually, an IA or a UX person that doesn’t care about analytics seems pretty phony to me, because they can’t prove what they preach. Analytics are easy, these people should be able to pick it up.
So what now? Well we can hope this gets better. UX and analytics are still pretty new fields (IA is a bit older) so I think they might evolve in the right direction, but we have to keep pushing so that things get done right. And don’t worry, if it doesn’t, I’ll complain again!
So yesterday I was talking with my friend that does iPhone applications about what data he gets from Apple about the applications he builds. It turns out that aside from the number of sales, he doesn’t have any data. I found this weird and we started talking about how it wouldn’t be that hard to build a kind of Google Analytics for Mobile Applications. In a sense, it would be very similar to the library GA for Flash except that you build a library for every Mobile SDK plus you have a webserver where the data is analysed.
It took so much time for a library of analytics to be made for Flash, I thought there might be a chance that nobody did anything like this for mobiles. So we got all excited (like so many other time), we started thinking how we would build this. But today I searched on the web and found Flurry a company that has an analytics division that does exactly this. Well not exactly how I would do it but about 85% the same. So my bubbles is a bit busted.
Flurry does it mostly right but their interface is a bit complicated and they didn’t make their analytics that specific to mobiles. There are plenty of concepts that exist in the mobile world that are new: what people do in their first and last run of the application, the number of tap (click with fingers) by session, the accelerometer, etc. And they don’t track that, yet. Also they provide an API for events, but not for navigation (pageViews in the GA world). I think navigation still has an important role in the analytics of an application than in the analytics of a website. You want to know what the users did in a certain section of your app (how many taps in the help section for example).
So all of that could be implemented and would give a better service than what Flurry is offering. The problem is that their platform is already built and even if there is not much competition (they seem to be the only ones doing this), it would still be hard to beat their momentum.
So what do you think? Should I invest time in this project knowing these risks, or should I let this go and wait for another idea?