Key Takeaways

  • HawkEye 360, a satellite radio-frequency (RF) data analytics company, had data siloed across teams and AWS accounts with no standardized access or tagging strategy, creating governance issues and blocking analysts from querying data independently.
  • Effectual built a governed AWS data lake on S3, using AWS Lambda and AWS Glue for data/transformation pipelines and AWS Lake Formation to enforce security, access, and compliance.
  • Effectual also built a custom web front-end so HawkEye's internal analysts could query the data lake through a graphical interface, without needing to write S3 queries themselves.
  • The project delivered a minimum viable product in just 10 weeks.
  • The new self-service data lake reduced HawkEye's reliance on expensive data scientists and engineers and gave the business new agility to launch products and revenue streams faster.

About HawkEye 360

HawkEye 360  (“HawkEye”) is the world’s leading radio frequency (RF) data analytics company. Using a novel commercial satellite constellation to identify, process and geolocate RF signals, the company fuses its unique satellite data with supplementary data sources to create powerful analytics products. Commercial, public-sector and international entities leverage HawkEye’s services for maritime domain awareness, defense planning, spectrum mapping, environmental ecosystem protection, and emergency beacon location, among other uses.

The Challenge

HawkEye was confronted with an escalating volume of siloed data across different teams and AWS accounts without a standardized access or tagging strategy. This led to data governance issues and limited the company’s analysts’ ability to access and query data independently. HawkEye sought an innovative solution that could handle varied data types from multiple sources, enable quick data analysis for valuable insights, and be accessible across the company irrespective of technical expertise. 

The Project

Effectual began by documenting and designing processes to ensure that HawkEye’s data is effectively captured, tagged and cataloged before it enters the new data lake. After assessing HawkEye’s data files and sources, Effectual designed and built a data lake architecture for AWS via S3 and developed data and transformation processes using AWS Lambda functions and AWS Glue. AWS Lake Formation was used to govern the data lake and handle all data security and compliance requirements.

Effectual tagged and cataloged HawkEye’s data and then built a custom web front-end that enables HawkEye’s internal analysts to query data via a graphical interface while eliminating the complex need of writing queries against S3.

Effectual designed and built a data lake architecture for AWS via S3 and developed data and transformation processes using AWS Lambda functions and AWS Glue. AWS Lake Formation was used to govern the data lake and handle all data security and compliance requirements.

The Result

In 10 weeks, Effectual delivered a minimum viable product for the data lake. The automated processes provided HawkEye with a reliable data source, surpassing data governance and compliance standards. This new solution reduced reliance on expensive data scientists and engineers, empowering HawkEye’s data analysts to independently execute queries and retrieve data, leading to valuable insights for potential customer use cases.  Consequently, HawkEye gained increased business agility and the capacity to introduce new products and revenue streams, significantly accelerating their ideation-to-product process. 

Frequently Asked Questions

What data challenge was HawkEye 360 facing before this project?

HawkEye's data was siloed across different teams and AWS accounts with no standardized access or tagging strategy, creating governance issues and limiting analysts' ability to query data independently.

What kind of AWS architecture did Effectual build for HawkEye?

A governed data lake on Amazon S3, with AWS Lambda and AWS Glue handling data and transformation processes and AWS Lake Formation enforcing data security, access, and compliance.

How did Effectual make the data lake usable for non-technical analysts?

Effectual built a custom web front-end that lets HawkEye's internal analysts query the data lake through a graphical interface, eliminating the need to write complex queries directly against S3.

How long did the project take to deliver?

Effectual delivered a minimum viable product for the data lake in 10 weeks.

What business impact did the new data lake have for HawkEye?

It reduced reliance on expensive data scientists and engineers, let analysts independently retrieve insights, and gave HawkEye more business agility to introduce new products and revenue streams faster.