Visit the post for more.
Overview
The article discusses the upcoming @Scale 2015 conference, where engineers from major tech companies will share insights on building scalable systems. Topics include data systems, source control management, and mobile app development for unreliable networks.
What You'll Learn
1
How to implement Heron for stream data processing at scale
2
Why using Spark Streaming can enhance batch processing in cloud environments
3
When to leverage GraphQL for efficient data fetching in applications
Key Questions Answered
What are the key topics discussed at the @Scale 2015 conference?
The @Scale 2015 conference will cover various topics including in-house data systems, source control management, IPv6, social graph caches, and mobile app development for unreliable networks. These discussions will feature insights from leading engineering teams at companies like Facebook, Google, and Uber.
How does Airbnb manage its data infrastructure in AWS?
Airbnb runs its production site and data infrastructure entirely within AWS. Paul Yang will discuss the company's migration to a two HDFS/Hive cluster setup, which allows them to manage multiple petabytes of data effectively across clusters.
What challenges does Facebook face in monitoring systems at scale?
Facebook's monitoring system faces challenges such as anomaly detection at scale and driving data exploration. Ostap Korkuna will delve into these issues and discuss the current strategies his team employs to tackle them.
What innovative mobile strategies are being discussed at the conference?
The conference will feature discussions on building mobile apps for unreliable networks, optimizing UIs, and transitioning mobile strategies to prioritize SDKs and platform components, as highlighted by various speakers from Twitter, Facebook, and Box.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Stream Processing
Heron
Used as the primary stream data processing engine at Twitter.
Data Processing
Hadoop
Utilized by Pinterest for serving large data sets.
Data Processing
Spark Streaming
Employed by Microsoft for batch processing in Office 365.
API
Graphql
Discussed by Facebook for efficient data fetching.
Networking
Ipv6
Discussed by Facebook regarding its growth and implications for developers.
Key Actionable Insights
1Consider implementing Heron as your stream data processing engine to handle real-time data efficiently.Heron has become the de facto standard at Twitter for stream processing, and its implementation can significantly improve data handling capabilities in production environments.
2Leverage Spark Streaming for batch processing to enhance the performance of your applications.Microsoft's experience with Spark Streaming in Office 365 showcases its effectiveness in handling real-time requests and processing large volumes of data.
3Adopt GraphQL to streamline data fetching and improve client-side performance.Facebook's exploration of GraphQL servers highlights the benefits of reducing over-fetching and under-fetching of data, which can lead to more efficient applications.