The Force Awakens and Big Data Management

The Force Awakens and Big Data Management

There is no bigger movie than Star Wars: The Force Awakens. We waited for it, imagined it, experienced it, saw it again, and some of us again and again.

I work in the data management space. Recently, I’ve spent a lot of time pondering the challenges and the opportunities of cohesive Big Data Management. As a result, as I watched The Force Awakens, I couldn’t help but notice the Big Data Management themes woven through the narrative. In fact, though it occurred “A long time ago,” the ENTIRE movie encapsulates the Big Data Management challenges of the modern era.

Big Data Management

As data proliferation increases exponentially, the data that organizations need for success is increasingly incomplete, inconsistent and unprotected. For these reasons, an effective Big Data Management platform must deliver in three core areas:

1. Big Data Integration
2. Big Data Quality
3. Big Data Security

The central narrative of The Force Awakens involves all three of these challenges, and covers all three core areas.

Big Data Integration

According to Gartner, through 2017, 90% of the information assets from big data analytic efforts will be “siloed and unleverageable”. This is the exact problem that both the light side and dark side face in The Force Awakens. Because of siloed data, they are unable to fully leverage the subset of the data that they have.

The key issue in the film, the goal of both the heroes and the villains, is to locate Luke Skywalker (both sides have pieces of the data set needed to complete this objective). The First Order has a galactic map from the archives of the empire, and only lack the details of the quadrant containing Luke’s location.

“He’s carrying a section of a navigational chart. We have the rest, recovered from archives of the Empire. We need the last piece.” – Kylo Ren, Star Wars: The Force Awakens

The Resistance possesses only the smaller map of Luke’s location, a subset of the total data set. The heroes lack context of where this granular map resides in the larger data set. As C3PO says to General Leia:

General, I regret to inform you, but this map recovered from BB-8 is only partially complete. And even worse, it matches no charted system on record. We simply do not have enough information to locate Master Luke.” – C3PO, Star Wars: The Force Awakens

The two data sets are meant for integration.

The movie’s entire plot is a race to see which side can accomplish this big data integration project more quickly and effectively than the other.

Big Data Quality

Data quality concerns can introduce an insurmountable obstacle to an organization’s ability to make wise choices. According to Experian, 26% of organizations suspect their data to be inaccurate. This is true in The Force Awakens as well.

Big Data Quality

Both the First Order and the Resistance possess a subset of a galactic navigational chart. However, if either portion of the chart proves to be even partially inaccurate, it would become profoundly more difficult for either Kylo Ren or General Leia to locate the aquatic planet Ahch-To and reach Luke Skywalker.

The Resistance has such poor Data Quality in place, that they are not even aware that they also possess the larger portion of the total data set:

“It is very doubtful that R2 would have the rest of the map in his backup data.” – C3PO, Star Wars: The Force Awakens

Of course, as it turns out, R2-D2 indeed does have the rest of the navigational chart in his backup data after all! An effective, Resistance-wide, Big Data Management platform would have exposed this profound Data Quality issue.

So where did R2-D2 get the larger portion of the galactic map? Some speculate R2 secretly downloaded while aboard the Death Star in Episode IV. If he did, just how current is the data? That would make it over 35 years old, right? Did it still contain Aldaraan, or had the Death Star DBAs cleaned their planetary archive list immediately after Alderaan was obliterated? One has to wonder, just how high is the quality of decades-old data?

Big Data Security

The largest worry of modern Information Technology workers is not knowing where sensitive data resides. This is a key concern for both sides in The Force Awakens. The Resistance doesn’t know they possess the large galactic map, within R2-D2. The First Order doesn’t know the location of the smaller map, to Ahch-To and Luke Skywalker.

This smaller map, the most important data set in the universe, begins with a very hackable security protocol. The film begins in a primitive hut, as an old explorer, Lor San Tekka, hand-delivers the sensitive data to Poe Dameron in a mysterious sack (This is the 21st-century equivalent of having the only copy of a nation’s nuclear launch codes on a single thumb drive). From there, the Data Security protocol becomes increasingly sketchy. First, Poe inserts the data set into the multi-reader of his astromech droid, BB-8. From there, BB-8 rolls through the desert, trying to avoid capture by vagabonds and junk dealers. The number of characters wanting to hack this precious data set include the parts scrapper Teedo, the junkyard dealer Unkar Plutt, the space gang leader Bala-Tik and of course, Kylo Ren himself.

Big Data Security

To enhance the data security, the light side needs to secure the data at the source. Rey herself becomes a new data security layer, personally protecting BB-8 from endless adversaries. First, Rey uses strong words to stop Teedo. She then uses her staff to stop Plutt’s goons. Last, she uses hungry Rathtars (accidentally) to stop Bala-Tik. Finally, once the sensitive data set (the map) has been transferred directly from BB8 to her (within her mind), she uses her new-found command of the force to stop Kylo Ren from taking it.

“I’m not giving you anything!” – Rey, Star Wars: The Force Awakens

Because of Rey, the data is secure at the source.

Episode VIII

We’ll have to wait until December 15, 2017 to learn whether or not the Resistance and the First Order improve their respective Big Data Management strategies and implement platforms to increase their chances of success.

However, you don’t need to wait another moment. You can implement a successful Big Data Management strategy right now, even today. Now you know, so as you go, May the Force be with you!

This post was originally published on the Informatica blog. Note that all characters and images from Star Wars: The Force Awakens are the property of Disney and LucasFilm.