It’s no longer news that Big Data is a big topic in transportation. Many people in our industry have been exploring how to use Big Data for years. But the technology landscape is evolving quickly, and in ways that may drive more widespread adoption of this type of data. In this post, I’ll share the four most important trends in Big Data to watch in 2018.
1. More and More Data Options
Companies like StreetLight and our peers are no longer all in a race to get access to the best data source. Because there’s TONS of data out there.
Here at StreetLight, we are approached nearly every month with a new provider of Big Data who wants to partner with us. They come both from different types of technologies as well as different companies within the same technology category. Examples include new apps that track locational patterns, fitness devices, apps dedicated to small freight operational management, IoT sensors, traffic cameras and more.
As we’ve always said – the best provider for transportation practitioners is one who allows you to mix and match different data, applying the right source to the problem at hand. There are simply way more options for raw data sources available than ever before. The ability of Big Data analytics providers to not only provide optionality, but also to recommend the right data source for the specific question at hand is going to increase adoption.
At StreetLight, we currently provide our clients with on-demand access to data analytics based on these data sources: Navigation-GPS from personal vehicles, Navigation-GPS from commercial trucks, and Location-Based Services from smartphone apps. We also help guide you to chose the right type of data for your analysis - and choosing the right data source can have a major impact on your results. For example, if you want to do an origin-destination study of personal vehicles for TAZs with trip purpose, you should use Location-Based Services Data. If you want to understand truck turning patterns near the port, use navigation-GPS data from commercial trucks.
2. Location-Based Services Data is Dominating
Despite the wealth of data options mentioned above, 2017 was certainly the year of Location-Based Services Data. 2018 will continue that trend.
Location-Based Services (LBS) Data is essentially location records collected from smart phone apps that track users’ locations by pinging in the background. (We discuss Location-Based Services Data a ton on our blog - check out this article on its benefits for rural areas.)
Here at StreetLight, we began our transition from cellular tower data to LBS Data in late 2016. We felt that the LBS had all the positive attributes we wanted – great sample size (nearly ¼ of the US and Canadian adult populations), the ability to normalize and accurately infer demographics, and excellent spatial precision. Cellular tower data has some good attributes, but its spatial precision in particular is lacking. Plus, it’s expensive and cumbersome to extract.
We selected our LBS data supplier, Cuebiq, after a thorough review of sources. Cuebiq’s Big Data sample (and thus our own) has grown dramatically since we started working together, and other apps and aggregators are popping up too. As a result, we think that cost of LBS data will go down per unit of data moving forward. As LBS data supply goes up and up and cost stays low, no cellular-based data analytics provider will be able to compete on a cost perspective. The fundamentals of installations in switches, the technology to try to eek out a little more precision from triangulation, and the fundamental costs of dealing with a few large telcos are just too expensive. In addition, we think that LBS’s flexibility and greater spatial precision will lead to better results than cellular-tower derived data, no matter how many clever data scientists you throw at it.
3. Data Fatigue is Becoming a Problem
Transportation engineers are getting overwhelmed with Big Data.
Lots of things come with the buzzword of Big Data attached – and the pace is just accelerating. At StreetLight, we provide just one class of data for a relatively narrow purpose: transportation planning. But agencies deal with the concept of Big Data in many ways, such as managing their own internal data, data from IoT sensors, etc.
Our clients are consistently starting to say that there are too many options and too many systems to keep track of. I think the winners in the Big Data landscape will be the providers that learn to cooperate and offer one-stop, integrated platforms to clients. We look forward to working with many other data suppliers on integration projects like this. We need to do more, as an industry, to allow our clients to bring all types of data sets together holistically. That’s what ultimately makes all the pieces of information government agencies have useful and informative.
4. Everyone Wants to Put the Data to Work
Data is just a cost until you use it.
I heard this phrase and variations of it several times at the Transportation Research Board Annual Meeting earlier in January. In an nutshell, it doesn't matter how much data you have - it's not productive until it becomes the basis for a decision. The most important thing for transportation planners, modelers and engineers to do is put the data to work. A critical part of this is not just having a pile of data: It’s having data in conjunction with the software to make it useful. The software is not a trivial add-on. It’s the key ingredient transforming transportation planning via Big Data.
Combing software with Big Data resources is exactly what we do at StreetLight. Our StreetLight InSight® platform lets you turn Big Data into actionable analytics on-demand. These analytics are designed from the ground up to solve transportation planning, modeling, and engineering challenges.
For example, we have several clients who have gotten our Regional Subscription which allows for unlimited analyses in our StreetLight InSight platform for a fixed annual fee. That means that anyone in the agency who wants data can just run a project in a few minutes – with no procurement, no budget discussion, no long compilation process. We’ve seen an explosion of pent-up demand with clients who have this type of contract. We estimate that our Regional Subscription clients are running over 10x the number of analytics that our clients who use our platform on a case-by-case basis. Putting the data to work also means that our customers want to learn about new use cases, they want to hear about each others’ best practices - and they want to start basing real decisions on Big Data.
What This All Means for 2018
It's clear by now that the future of this industry won’t be about competing over who has the biggest data. The future will belong to people who know how to tell stories with Big Data. And that story can’t be “oh what a cool map.” It has to be a story that leads to taking action. That's why our theme of 2018 at StreetLight Data is “Putting Big Data to Work.”
Note: This blog post is based on remarks I gave at a panel discussing during the Transportation Research Board Annual Meeting 2018. The panel was named "User Demand and Big Data Supply". Thanks to Krishnan Viswanathan from Cambridge Systematics and Guy Rousseau from the Atlanta Regional Commission for convening and facilitating such a timely, diverse, and lively discussion. I would also like to thank to all the people who attended the panel – it was standing room only with people being turned away at the door!