The LA Times issues weekly “crime alerts” for any neighborhood with an “unusual jump” in reported crimes. An “unusual jump” means that reported crimes during the past week must be “up significantly” vs. the average week during the past quarter.
What is this supposed to tell us? The implication seems to be that a neighborhood’s crime is on the rise. That is what most readers would probably think. But that is not what their approach does. It simply grabs statistical fluctuations and then reports them in a post “automatically created by an algorithm”. (*)
In order to have meaning the numbers must be compared to historical patterns. Have past spikes in reported crime been followed by a sustained rise? That question can be answered using only the reported numbers and statistical techniques. It would be a big improvement on the current approach. But to really have meaning, there needs to be context to both the recent numbers and the historical analysis.
For example, perhaps sometimes in the past a spike does seem to predict a coming rise in crime, while in other instances it does not, yet the historical pattern does not fit a random distribution. Something is going on. The numbers can’t answer that. Perhaps when police activity increases in response to a spike, the crime rate quickly reverts to normal (and the activity may shift to other areas). The crime report numbers can’t tell you that. Talking to cops and experts could be reveal information you simply cannot get from number crunching this data set.
Similarly, we don’t know if and when the perpetrators of the recent and historical crimes were caught and taken off the streets. If we know a gang of burglars has been jailed, then unless a new crew steps right in, the recent crime spike caused by that gang cannot inform us about next week.
Note: Perhaps there is some rudimentary statistics used in the algorithm. For example, perhaps the neighborhood’s weekly crime must be up “2 sigmas”. But the authors say they only compare to the most recent quarter, which contains about 13 weeks, so meaningful statistical properties could not be calculated. Furthermore, different kinds of serious crime may have different patterns – some may be random while others show distinct patterns.
Here’s their text describing their approach
The Times’ crime alerts detect unusual jumps in crime reports in neighborhoods. Each time Crime L.A. gets new reports — usually three to five times a week — they are compared to what has been typical for that neighborhood over the past quarter. An alert is generated if violent and/or property crime reports from the most recent week are up significantly over the average. In order to qualify for an alert, the jump in crime reports must significantly surpass the average week. In addition, a minimum of three crimes must be reported.
* I’ll have more to say about that in another post about real expertise vs. blind number crunching.