How LinkedIn Uses Azure

The article discusses the modernization of the LDAP and Kerberos infrastructure that secures Hadoop at LinkedIn, detailing the transition from a legacy setup to a highly available, automated system...

The article discusses the development of domain-adapted foundation GenAI models at LinkedIn, focusing on their application within the Economic Opportunity Network (EON) project.

The article discusses the implementation of an automated GenAI-driven search quality evaluation system for LinkedIn's typeahead suggestions.

The article discusses LinkedIn's transition to Azure Linux as its primary operating system, detailing the motivations, challenges, and strategies involved in the migration.

The article discusses the development of an analytics dashboard for LinkedIn Group admins, providing real-time insights into group growth and engagement metrics.

GrafLI is a cloud-native monitoring and visualization platform developed by LinkedIn to enhance the developer experience and increase engineering velocity in Azure environments.

The article discusses ValiData, a scalable automated config-driven data validation tool used at LinkedIn to ensure the accuracy and consistency of large datasets.

The article discusses LinkedIn's platform strategy in the context of Generative AI (GenAI), emphasizing how it enables rapid innovation and development.

The article discusses Costwiz, a tool developed by LinkedIn to optimize cloud costs on Azure by monitoring resource utilization and providing actionable insights.

The article discusses how LinkedIn scaled its Salt infrastructure to support its growing needs for remote execution jobs, achieving a tenfold increase in job capacity and improved reliability.

The article discusses LinkedIn's Responsible AI principles and their application in developing generative AI tools.

(Re)building Threat Detection and Incident Response at LinkedIn discusses the transformation of LinkedIn's security operations, specifically the Threat Detection and Incident Response team (SEEK) a...

Feathr, a feature store developed by LinkedIn, has joined the LF AI & Data Foundation, which supports open-source innovation in AI and data.

The article discusses Bef's journey at LinkedIn, highlighting his transition from a backend engineer to an engineering director while emphasizing the importance of mentorship and building an inclus...

The article discusses the implementation of Remote Direct Memory Access (RDMA) over Converged Ethernet version 2 (RoCEv2) in LinkedIn's data centers to enhance storage scalability and performance.

The article discusses the open sourcing of Feathr, LinkedIn's feature store designed to simplify machine learning feature management and enhance developer productivity.

The article introduces FastTreeSHAP, an open-source Python package designed to accelerate SHAP value computations for tree-based models.

The article discusses LinkedIn's migration to Azure Front Door (AFD) and the significant performance improvements achieved through this transition.

The article shares the career journey of Mounika, a senior backend engineer at LinkedIn, highlighting her transition from South India to the U. S.

The article discusses LinkedIn's transition from a proprietary analytics tech stack to an open-source big data technology stack, detailing the challenges faced and the improvements made during the ...

The article discusses LinkedIn's Distributed Firewall (DFW), a host-level firewalling system designed to manage extensive firewall rules across a vast network of hosts.

Project Magnet introduces push-based shuffle in Apache Spark 3. 2, enhancing shuffle scalability and reliability.

This article discusses the challenges and solutions LinkedIn faced while scaling its Hadoop YARN cluster beyond 10,000 nodes.

The article discusses the development of Microsoft-powered native video meetings on LinkedIn, emphasizing the integration of Azure Communication Services to enhance user experience.

This article discusses the evolution of LinkedIn's Daily Executive Dashboard (DED) from a simple dashboard to a robust enterprise-grade data pipeline.

The article discusses LinkedIn's significant advancements in scaling the Hadoop Distributed File System (HDFS), achieving the milestone of storing 1 exabyte of data and optimizing performance throu...

The article introduces Magnet, a scalable and performant shuffle architecture designed for Apache Spark, addressing the challenges faced in shuffle operations at LinkedIn.

The article discusses the architectural improvements made to Jhubbub, LinkedIn's internal backend service for processing RSS feeds, by leveraging Apache Helix to create a stateless and elastic syst...

The article discusses a significant incident involving data loss in a Hadoop ecosystem at LinkedIn, highlighting the lessons learned from the event.

The article discusses LinkedIn's migration of its edge infrastructure to Azure Front Door (AFD) to enhance scalability and improve user experience.

Apache Pinot 0. 3. 0 is an open-source, distributed OLAP data store developed at LinkedIn, designed for near-real-time analytics.

The article discusses LinkedIn's open sourcing of DataHub, a metadata search and discovery platform, detailing its development journey from WhereHows to DataHub.

The article discusses the process of upgrading LinkedIn's Pinot servers from Red Hat Enterprise Linux 6 (RHEL6) to RHEL7 with minimal service interruptions.

The article highlights the top ten engineering blogs from LinkedIn in 2019, focusing on popular topics such as open source, artificial intelligence, and technical challenges at scale.

The article discusses LinkedIn's journey towards automating load testing to eliminate manual toil, enhance efficiency, and ensure reliable service during peak traffic.

The article discusses LinkedIn's 4th annual SRE[in]con conference, which brought together over 700 site engineers and partners for a series of workshops and talks.

The article discusses LinkedIn's ongoing efforts to enhance its infrastructure to support its growing user base and operational demands.

The article discusses the open-sourcing of Brooklin, a distributed service for near real-time data streaming at scale, which has been in production at LinkedIn since 2016.

The article discusses the entry of Pinot into the Apache Incubator, highlighting its capabilities as a scalable distributed OLAP data store developed at LinkedIn.

The article discusses the rebuilding of LinkedIn's Profile Highlights System, focusing on the transition from a monolithic architecture to a microservices-based plug-in architecture.

The article discusses a community meetup held at LinkedIn focused on Apache Hadoop, highlighting contributions from various organizations and key presentations on topics like TensorFlow on YARN, Ha...

The article announces the release of Samza 1. 0, a distributed stream processing framework developed at LinkedIn, highlighting its significant features and improvements.

The article provides an introduction to the role of artificial intelligence (AI) at LinkedIn, detailing how it is integrated into various products and services to enhance user experience and operat...

The article discusses Nuage, LinkedIn's cloud management team, which is designed to scale data systems management effectively.

The article discusses the integration of LinkedIn's Universal Content Filtering (UCF) platform with Microsoft's Content Moderator service to enhance the detection of inappropriate content on Linked...

The article discusses the implementation of dynamic machine translation in the LinkedIn feed, addressing the challenges of language barriers among users.

The article discusses the integration of Project Falco into the SONiC community, highlighting its significance in building a programmable data center fabric on an open network operating system.

The article discusses Brooklin, a data ingestion service developed by LinkedIn to facilitate streaming data from various sources to multiple destinations.

This article discusses the challenges of data access in high-scale stream processing, particularly focusing on the read/write and read-only data access patterns.

The article discusses the challenges faced in stream processing, particularly focusing on the limitations of the Lambda architecture.