Case Studies

Import big data into AWS as fast as possible

Industry

Product name

Client

Scroll down

A quick look

Our client needed to import big data as fast as possible but at the same time ensure enough cloud infrastructure capacity to run the client's e-commerce website and mobile app.

Implementation schema

Our Mission

Build a scaling mechanism to spin-up or kill-off Fargate ECS services processing Messaging Queues.

Our Approach

Our solution was a Lambda function which scales the ECS services processing the Messaging Queues. The Lambda function is based on an algorithm that calculates the available IOPS of the database, in order to make sure that the ECS services process the Messaging Queues as fast as possible, without causing any interruption in the availability of the web page.

Outcomes

  • Easy communication between the Lambda and ECS via the Python AWS SDK
  • The cloudwatch event to trigger the Lambda in a desired time period
  • Prioritization of the ECS tasks via a configuration file in the Lambda. This was impossible to achieve with a native ECS cron solution for example.
  • Accurate calculation of the available RDS IOPS

More of our work

NYC's real estate platform for apartments & brokers

Igluu is a community-based real estate platform focused on seamless search and discovery. Unlike other sites, Igluu protects broker's exclusive listings, never auctioning their leads to random un-vetted agents.

See more

Storing solar power without batteries

Industry 4.0 system that integrates data from solar power connected IoT devices, measuring whether a user is removing energy from or putting it back in the network

See more