Submit     Newsletter     Search     Register  
Software Engineering Daily on Smash Notes

Software Engineering Daily podcast.

December 28, 2019

Technical interviews about software topics.



Episodes with Smash Notes

AWS has over 150 different services. Databases, log management, edge computing, and lots of others. Instead of being overwhelmed by all of these products, an engineering team can simplify their workflow by focusing on a small subset of AWS services–the defaults. Daniel Vassalo is the author of The Good Parts of AWS. An excerpt from


The post The Good Parts of AWS with Daniel Vassallo appeared first on Software Engineering Daily.

The modern release workflow involves multiple stakeholders: engineers, management, designers, and product managers. It is a collaborative process that is often held together with brittle workflows. A developer deploys a new build to an ad hoc staging environment and pastes a link to that environment in Slack. Other stakeholders click on that link, then send


The post Pull Request Environments with Eric Silverman appeared first on Software Engineering Daily.

Deepgram is an end-to-end deep learning platform for speech recognition. Unlike the general purpose APIs from Google or Amazon, Deepgram models are custom-trained for each customer. Whether the customer is a call center, a podcasting company, or a sales department, Deepgram can work with them to build something specific to their use case. Sound data


The post Deepgram: End-to-End Speech Recognition with Scott Stephenson appeared first on Software Engineering Daily.

DynamoDB is a managed NoSQL database service from AWS. It is widely used as a transactional database to fulfill key-value and wide-column data models. In a previous show with Rick Houlihan, we explored how to build a data model and optimize the query patterns for a NoSQL database.  Today’s show is about DynamoDB specifically: partitioning,


The post DynamoDB with Alex DeBrie appeared first on Software Engineering Daily.

As a user browses a webpage, that browser session generates events that need to be recorded, validated, enriched, and stored. This data is sometimes called customer data infrastructure, or CDI. This data requires a full stack of different tools: a system on the frontend to collect the data, middleware to transport the data, and backend


The post Snowplow Analytics: Data Collection Platform with Alex Dean appeared first on Software Engineering Daily.

A software company manages and interacts with hundreds of APIs. These APIs require testing, performance analysis, authorization management, and release management. In a word, APIs require collaboration. Postman is a system for API collaboration. It allows users to test APIs with collections of requests, monitor the API responses, and visualize the query results. Users of


The post Postman: API Development with Abhinav Asthana appeared first on Software Engineering Daily.

At a customer service center, thousands of hours of audio are generated. This audio provides a wealth of information to transcribe and analyze. With the additional data of the most successful customer service representatives, machine learning models can be trained to identify which speech patterns are associated with a successful worker. By identifying these speaking


The post Cresta: Speech ML for Calls with Zayd Enam appeared first on Software Engineering Daily.

Originally published July 6, 2017. We are taking a few weeks off. We’ll be back soon with new episodes. React Native allows developers to reuse components from one user interface on multiple platforms. React Native was introduced by Facebook to reduce the pain of teams who were rewriting their user interfaces for web, iOS, and


The post React Native Ecosystem with Nader Dabit (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published October 8, 2019. We are taking a few weeks off. We’ll be back soon with new episodes. Video surveillance impacts human lives every day.  On most days, we do not feel the impact of video surveillance. But the effects of video surveillance have tremendous potential. It can be used to solve crimes and


The post Traces: Video Recognition with Veronica Yurchuk and Kostyantyn Shysh (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published July 25, 2019. We are taking a few weeks off. We’ll be back soon with new episodes. Envoy is an open source edge and service proxy that was originally developed at Lyft.  Envoy is often deployed as a sidecar application that runs alongside a service and helps that service by providing features such


The post Envoy Mobile with Matt Klein (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published May 2, 2017. We are taking a few weeks off. We’ll be back soon with new episodes. A new programmer learns to build applications using data structures like a queue, a cache, or a database. Modern cloud applications are built using more sophisticated tools like Redis, Kafka, or Amazon S3. These tools do


The post Data Intensive Applications with Martin Kleppman (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published December 20, 2019. We are taking a few weeks off. We’ll be back soon with new episodes. freeCodeCamp was started five years ago with the goal of providing free coding education to anyone on the Internet. freeCodeCamp has become the best place to begin learning how to write software. There are many other


The post freeCodeCamp with Quincy Larson (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published April 14, 2017. We are taking a few weeks off. We’ll be back soon with new episodes. Facebook’s open source projects include React, GraphQL, and Cassandra. These projects are key pieces of infrastructure used by thousands of developers–including engineers at Facebook itself. These projects are able to gain traction because Facebook takes time


The post Facebook Open Source with Tom Occhino (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published October 24, 2019. We are taking a few weeks off. We’ll be back soon with new episodes. Redis is an in-memory database that persists to disk. Redis is commonly used as an object cache for web applications. Applications are composed of caches and databases. A cache typically stores the data in memory, and


The post Redis with Alvin Richards (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published November 21, 2019. We are taking a few weeks off. We’ll be back soon with new episodes. HTTP is a protocol that allows browsers and web applications to communicate across the Internet. Everyone knows that HTTP is doing some important work, because “HTTP” is at the beginning of most URLs that you enter


The post HTTP with Julia Evans (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published June 13, 2019. We are taking a few weeks off. We’ll be back soon with new episodes. Machine learning allows software to improve as that software consumes more data. Machine learning is a tool that every software engineer wants to be able to use. Because machine learning is so broadly applicable, software companies


The post Stripe Machine Learning Infrastructure with Rob Story and Kelley Rivoire (Summer Break Repeat) appeared first on Software Engineering Daily.

Originally published January 31, 2019 Artificial intelligence is reshaping every aspect of our lives, from transportation to agriculture to dating. Someday, we may even create a superintelligence–a computer system that is demonstrably smarter than humans. But there is widespread disagreement on how soon we could build a superintelligence. There is not even a broad consensus


The post Architects of Intelligence with Martin Ford Holiday Repeat appeared first on Software Engineering Daily.

Cruise is an autonomous car company with a development cycle that is highly dependent on testing its cars–both in the wild and in simulation. The testing cycle typically requires cars to drive around gathering data, and that data to subsequently be integrated into a simulated system called Matrix. With COVID-19, the ability to run tests


The post Cruise Simulation with Tom Boyd appeared first on Software Engineering Daily.

Grafana is an open source visualization and monitoring tool that is used for creating dashboards and charting time series data. Grafana is used by thousands of companies to monitor their infrastructure. It is a popular component in monitoring stacks, and is often used together with Prometheus, ElasticSearch, MySQL, and other data sources. The engineering complexities


The post Grafana with Torkel Ödegaard appeared first on Software Engineering Daily.

Apache Airflow was released in 2015, introducing the first popular open source solution to data pipeline orchestration. Since that time, Airflow has been widely adopted for dependency-based data workflows. A developer might orchestrate a pipeline with hundreds of tasks, with dependencies between jobs in Spark, Hadoop, and Snowflake. Since Airflow’s creation, it has powered the


The post Apache Airflow with Maxime Beauchemin, Vikram Koka, and Ash Berlin-Taylor appeared first on Software Engineering Daily.

The life cycle of data management includes data cleaning, extraction, integration, analysis and exploration, and machine learning models. It would be great if all of this data management could be handled with automation, but unfortunately that is not an option. For most applications, data management requires a human in the loop. A human in the


The post Human in the Loop Data Analytics with Aditya Parameswaran appeared first on Software Engineering Daily.

Kubernetes continues to mature as a platform for infrastructure management. At this point, many companies have well-developed workflows and deployment patterns for working with applications built on Kubernetes. The complexity of some of these deployments may be daunting, and when a new employee joins a company, that employee needs to get quickly onboarded with the


The post Tilt: Kubernetes Tooling with Dan Bentley appeared first on Software Engineering Daily.

Uber needs to visualize data on a range of different surfaces. A smartphone user sees cars moving around on a map as they wait for their ride to arrive. Data scientists and operations researchers within Uber study the renderings of traffic moving throughout a city. Data visualization is core to Uber, and the company has


The post Uber’s Data Visualization Tools with Ib Green appeared first on Software Engineering Daily.

A frontend developer issuing a query to a backend server typically requires the developer to issue that query through an ORM or a raw database query. Prisma is an alternative to both of these data access patterns, allowing for easier database access through auto-generated, type-safe query building tailored to an existing database schema. By integrating


The post Prisma: Modern Database Tooling with Johannes Schickling appeared first on Software Engineering Daily.

Machine learning workflows have had a problem for a long time: taking a model from the prototyping step and putting it into production is not an easy task. A data scientist who is developing a model is often working with different tools, or a smaller data set, or different hardware than the environment which that


The post Tecton: Machine Learning Platform from Uber with Kevin Stumpf appeared first on Software Engineering Daily.

Many data sources produce new data points at a very high rate. With so much data, the issue of data quality emerges. Low quality data can degrade the accuracy of machine learning models that are built around those data sources. Ideally, we would have completely clean data sources, but that’s not very realistic. One alternative


The post HoloClean: Data Quality Management with Theodoros Rekatsinas appeared first on Software Engineering Daily.

Server infrastructure traditionally consists of monolithic servers containing all of the necessary hardware to run a computer. These different hardware components are located next to each other, and do not need to communicate over a network boundary to connect the CPU and memory. LegoOS is a model for disaggregated, network-attached hardware. LegoOS disseminates the traditional


The post Disaggregated Servers with Yiying Zhang appeared first on Software Engineering Daily.

Kubernetes has become a highly usable platform for deploying and managing distributed systems.  The user experience for Kubernetes is great, but is still not as simple as a full-on serverless implementation–at least, that has been a long-held assumption. Why would you manage your own infrastructure, even if it is Kubernetes? Why not use autoscaling Lambda


The post Kubernetes vs. Serverless with Matt Ward appeared first on Software Engineering Daily.

Every software company is a distributed system, and distributed systems fail in unexpected ways.  This ever-present tendency for systems to fail has led to the rise of failure testing, otherwise known as chaos engineering. Chaos engineering involves the deliberate failure of subsystems within an overall system to ensure that the system itself can be resilient


The post Distributed Systems Research with Peter Alvaro appeared first on Software Engineering Daily.

Brex is a credit card company that provides credit to startups, mostly companies which have raised money. Brex processes millions of transactions, and uses the data from those transactions to assess creditworthiness, prevent fraud, and surface insights for the users of their cards. Brex is full of interesting engineering problems. The high volume of transactions


The post Brex Engineering with Cosmin Nicolaescu appeared first on Software Engineering Daily.

Devices on the edge are becoming more useful with improvements in the machine learning ecosystem. TensorFlow Lite allows machine learning models to run on microcontrollers and other devices with only kilobytes of memory. Microcontrollers are very low-cost, tiny computational devices. They are cheap, and they are everywhere. The low-energy embedded systems community and the machine


The post Edge Machine Learning with Zach Shelby appeared first on Software Engineering Daily.

For the last five months, we have been working on a new version of Software Daily, the platform we built to host and present our content.  We are creating a platform that integrates the podcast with a set of other features that make it easier to learn from the audio interviews.  Software Daily includes the


The post Software Daily appeared first on Software Engineering Daily.

Over the last 5 years, web development has matured considerably. React has become a standard for frontend component development. GraphQL has seen massive growth in adoption as a data fetching middleware layer. The hosting platforms have expanded beyond AWS and Heroku, to newer environments like Netlify and Vercel. These changes are collectively known as the


The post RedwoodJS with Tom Preston-Werner appeared first on Software Engineering Daily.

Geospatial analytics tools are used to render visualizations for a vast array of applications. Data sources such as satellites and cellular data can gather location data, and that data can be superimposed over a map. A map-based visualization can allow the end user to make decisions based on what they see. ArcGIS is one of


The post ArcGIS: Geographic Information Software with Max Payson appeared first on Software Engineering Daily.

Customer data infrastructure is a type of tool for saving analytics and information about your customers. The company that is best known in this category is Segment, a very popular API company. This customer data is used for making all kinds of decisions around product roadmap, pricing, and design. RudderStack is a company built around


The post RudderStack: Open Source Customer Data Infrastructure with Soumyadeb Mitra appeared first on Software Engineering Daily.

Matterport is a company that builds 3-D imaging for the inside of buildings, construction sites, and other locations that require a “digital twin.” Generating digital images of the insides of buildings has a broad spectrum of applications, and there are considerable engineering challenges in building such a system. Matterport’s hardware stack involves a camera built


The post Matterport 3-D Imaging with Japjit Tulsi appeared first on Software Engineering Daily.

There are many bad recipe web sites. Every time I navigate to a recipe website, it feels like my browser is filling up with spyware. The page loads slowly, everything seems broken, I can feel the 25 different JavaScript adtech tags interrupting each other. Whether I am searching for banana bread or a spaghetti sauce


The post Frontend Performance with Anycart’s Rafael Sanches appeared first on Software Engineering Daily.

For the last five months, we have been working on a new version of Software Daily, the platform we built to host and present our content.  We are creating a platform that integrates the podcast with a set of other features that make it easier to learn from the audio interviews.  Software Daily includes the


The post Software Daily appeared first on Software Engineering Daily.

Amazon’s virtual server instances have come a long way since the early days of EC2. There are now a wide variety of available configuration options for spinning up an EC2 instance, which can be chosen from based on the workload that will be scheduled onto a virtual machine. There are also Fargate containers and AWS


The post AWS Virtualization with Anthony Liguori appeared first on Software Engineering Daily.

A credit score is a rating that allows someone to qualify for a line of credit, which could be a loan such as a mortgage, or a credit card. We are assigned a credit score based on a credit history, which could be related to work history, rental payments, or loan repayments.  One problem with


The post International Consumer Credit Infrastructure with Brian Regan and Misha Esipov appeared first on Software Engineering Daily.

A large software company such as Dropbox is at a constant risk of security breaches. These security breaches can take the form of social engineering attacks, network breaches, and other malicious adversarial behavior. This behavior can be surfaced by analyzing collections of log data. Log-based threat response is not a new technique. But how should


The post Grapl: Graph-Based Detection and Response with Colin O’Brien appeared first on Software Engineering Daily.

Infrastructure-as-code tools are used to define the architecture of software systems. Common infrastructure-as-code tools include Terraform and AWS CloudFormation.  When infrastructure is defined as code, we can use static analysis tools to analyze that code for configuration mistakes, just as we could analyze a programming language with traditional static analysis tools. When a developer writes


The post Static Analysis for Infrastructure with Guy Eisenkot appeared first on Software Engineering Daily.

Social distancing has been imposed across the United States. We are running an experiment unlike anything before it in history, and it is likely to have a lasting impact on human behavior. By looking at location data of how people are moving around today, we can examine the real-world impacts of social distancing. SafeGraph is


The post Social Distancing Data with Ryan Fox Squire appeared first on Software Engineering Daily.

Dropbox is a consumer storage product with petabytes of data. Dropbox was originally started on the cloud, backed by S3. Once there was a high enough volume of data, Dropbox created its own data centers, designing hardware for the express purpose of storing user files.  Over the last 13 years, Dropbox’s infrastructure has developed hardware,


The post Dropbox Engineering with Andrew Fong appeared first on Software Engineering Daily.

“Data stream” is a word that can be used in multiple ways. A stream can refer to data in motion or data at rest.  When a stream is data in motion, an endpoint is receiving new pieces of data on a continual basis. Each new data point is sent over the wire and captured by


The post Pravega: Storage for Streams with Flavio Junquiera appeared first on Software Engineering Daily.

Redis is an in-memory object storage system that is commonly used as a cache for web applications. This core primitive of in-memory object storage has created a larger ecosystem encompassing a broad set of tools. Redis is also used for creating objects such as queues, streams, and probabilistic data structures. Machine learning systems also need


The post Advanced Redis with Alvin Richards appeared first on Software Engineering Daily.

For many applications, a transactional MySQL database is the source of truth. To make a MySQL database scale, some developers deploy their database using Vitess, a sharding system built on top of Kubernetes.  Jiten Vaidya and Anthony Yeh work at PlanetScale, a company that focuses on building and supporting MySQL databases sharded with Vitess. Their


The post Multicloud MySQL with Jiten Vaidya and Anthony Yeh appeared first on Software Engineering Daily.

We are all living in social isolation due to the quarantine from COVID-19. Isolation is changing our habits and our moods, ravaging the economy, and changing how we work. One positive change is that more people have been reconnecting with their friends and family over frequent calls and video chats. Isolation is not a normal


The post Isolation with Courtland Allen and Anurag Goel appeared first on Software Engineering Daily.

A data warehouse is a system for performing fast queries on large amounts of data. A data lake is a system for storing high volumes of data in a format that is slow to access. A typical workflow for a data engineer is to pull data sets from this slow data lake storage into the


The post Data Lakehouse with Michael Armbrust appeared first on Software Engineering Daily.

A content management system (CMS) defines how the content on a website is arranged and presented. The most widely used CMS is WordPress, the open source tool that is written in PHP. A large percentage of the web consists of WordPress sites, and WordPress has a huge ecosystem of plugins and templates. Despite the success


The post JAMStack Content Management with Scott Gallant, Jordan Patterson, and Nolan Phillips appeared first on Software Engineering Daily.

A data workflow scheduler is a tool used for connecting multiple systems together in order to build pipelines for processing data. A data pipeline might include a Hadoop task for ETL, a Spark task for stream processing, and a TensorFlow task to train a machine learning model.  The workflow scheduler manages the tasks in that


The post Prefect Dataflow Scheduler with Jeremiah Lowin appeared first on Software Engineering Daily.

A relational database often holds critical operational data for a company, including user names and financial information. Since this data is so important, a relational database must be architected to avoid data loss. Relational databases need to be a distributed system in order to provide the fault tolerance necessary for production use cases. If a


The post CockroachDB with Peter Mattis appeared first on Software Engineering Daily.

Python is the most widely used language for data science, and there are several libraries that are commonly used by Python data scientists including Numpy, Pandas, and scikit-learn. These libraries improve the user experience of a Python data scientist by giving them access to high level APIs. Data science is often performed over huge datasets,


The post Dask: Scalable Python with Matthew Rocklin appeared first on Software Engineering Daily.

Chatbots became widely popular around 2016 with the growth of chat platforms like Slack and voice interfaces such as Amazon Alexa. As chatbots came into use, so did the infrastructure that enabled chatbots. NLP APIs and complete chatbot frameworks came out to make it easier for people to build chatbots. The first suite of chatbot


The post Rasa: Conversational AI with Tom Bocklisch appeared first on Software Engineering Daily.

Serverless computing is a way of designing applications that do not directly address or deploy application code to servers. Serverless applications are composed of stateless functions-as-a-service and stateful data storage systems such as Redis or DynamoDB.  Serverless applications allow for scaling up and down the entire architecture, because each component is naturally scalable. And this


The post Cloudburst: Stateful Functions-as-a-Service with Vikram Sreekanti appeared first on Software Engineering Daily.

NGINX is a web server that can be used to manage the APIs across an organization. Managing these APIs involves deciding on the routing and load balancing across the servers which host them. If the traffic of a website suddenly spikes, the website needs to spin up new replica servers and update the API gateway


The post NGINX API Management with Kevin Jones appeared first on Software Engineering Daily.

Web development has historically had more work being done on the server than on the client. The observability tooling has reflected this emphasis on the backend. Monitoring tools for log management and backend metrics have existed for decades, helping developers debug their server infrastructure. Today, web frontends have more work to do. Detailed components in


The post Frontend Monitoring with Matt Arbesfeld appeared first on Software Engineering Daily.

Zoom video chat has become an indispensable part of our lives. In a crowded market of video conferencing apps, Zoom managed to build a product that performs better than the competition, scaling with high quality to hundreds of meeting participants, and millions of concurrent users. Zoom’s rapid growth in user adoption came from its focus


The post Zoom Vulnerabilities with Patrick Wardle appeared first on Software Engineering Daily.

Facebook applications use maps for showing users where to go. These maps can display businesses, roads, and event locations. Understanding the geographical world is also important for performing search queries that take into account a user’s location. For all of these different purposes, Facebook needs up-to-date, reliable mapping data. OpenStreetMap is an open system for


The post Facebook OpenStreetMap Engineering with Saurav Mapatra and Jacob Wasserman appeared first on Software Engineering Daily.

NGINX is a web server that is used as a load balancer, an API gateway, a reverse proxy, and other purposes. Core application servers such as Ruby on Rails are often supported by NGINX, which handles routing the user requests between the different application server instances.  This model of routing and load balancing between different


The post NGINX Service Mesh with Alan Murphy appeared first on Software Engineering Daily.

Shopify is a platform for selling products and building a business. It is a large e-commerce company with hundreds of engineers and several different mobile apps. Shopify’s engineering culture is willing to adopt new technologies aggressively, trying new tools that might provide significant leverage to the organization. React Native is one of those technologies. React


The post Shopify React Native with Farhan Thawar appeared first on Software Engineering Daily.

Ceph is a storage system that can be used for provisioning object storage, block storage, and file storage. These storage primitives can be used as the underlying medium for databases, queueing systems, and bucket storage. Ceph is used in circumstances where the developer may not want to use public cloud resources like Amazon S3. As


The post Ceph Storage System with Sage Weil appeared first on Software Engineering Daily.

Data analysts need to collaborate with each other in the same way that software engineers do. They also need a high quality development environment.  These data analysts are not working with programming languages like Java and Python, so they are not using an IDE such as Eclipse. Data analysts predominantly use SQL, and the tooling


The post Collaborative SQL with Rahil Sondhi appeared first on Software Engineering Daily.

When a developer spins up a virtual machine on AWS, that virtual machine could be purchased using one of several types of cost structures. These cost structures include on-demand instances, spot instances, and reserved instances. On-demand instances are often the most expensive, because the developer gets reliable VM infrastructure without committing to long-term pricing. Spot


The post Reserved Instances with Aran Khanna appeared first on Software Engineering Daily.

Machine learning models require the use of training data, and that data needs to be labeled. Today, we have high quality data infrastructure tools such as TensorFlow, but we don’t have large high quality data sets. For many applications, the state of the art is to manually label training examples and feed them into the


The post Snorkel: Training Dataset Management with Braden Hancock appeared first on Software Engineering Daily.

A workflow is an application that involves more than just a simple request/response communication. For example, consider a session of a user taking a ride in an Uber. The user initiates the ride, and the ride might last for an hour. At the end of the ride, the user is charged for the ride and


The post Cadence: Uber’s Workflow Engine with Maxim Fateev appeared first on Software Engineering Daily.

Kafka is a distributed stream processing system that is commonly used for storing large volumes of append-only event data. Kafka has been open source for almost a decade, and as the project has matured, it has been used for new kinds of applications.  Kafka’s pubsub interface for writing and reading topics is not ideal for


The post kSQLDB: Kafka Streaming Interface with Michael Drogalis appeared first on Software Engineering Daily.

Building a game is not easy. The development team needs to figure out a unique design and gameplay mechanics that will attract players. There is a great deal of creative work that goes into making a game successful, and these games are often built with low budgets by people who are driven by the art


The post Godot Game Engine with Juan Linietsky appeared first on Software Engineering Daily.

V8 is the JavaScript engine that runs Chrome. Every popular website makes heavy use of JavaScript, and V8 manages the execution environment of that code. The code that processes in your browser can run faster or slower depending on how “hot” the codepath is. If a certain line of code is executed frequently, that code


The post V8 Lite with Ross McIlroy appeared first on Software Engineering Daily.

Serverless tools have come a long way since the release of AWS Lambda in 2014. Serverless apps were originally architected around Lambda, with the functions-as-a-service being used to glue together larger pieces of functionality and API services. Today, many of the common AWS services such as API Gateway and DynamoDB have functionality built in to


The post Serverless Development with Jeremy Daly appeared first on Software Engineering Daily.

Cortico is a non-profit that builds audio tools to improve public dialogue. Allison King is an engineer at Cortico, and she joins the show to talk about the process of building audio applications.  One of these applications was a system for ingesting radio streams, transcribing the radio, and looking for duplicate information across the different


The post Audio Data Engineering with Allison King appeared first on Software Engineering Daily.

Facebook Messenger is a chat application that millions of people use every day to talk to each other. Over time, Messenger has grown to include group chats, video chats, animations, facial filters, stories, and many more features. Messenger is a tool for utility as well as for entertainment. Messenger is used both on mobile and


The post Facebook Messenger Engineering with Mohsen Agsen appeared first on Software Engineering Daily.

Modern web development involves a complicated toolchain for managing dependencies. One part of this toolchain is the bundler, a tool that puts all your code and dependencies together into static asset files. The most popular bundler is webpack, which was originally released in 2012, before browsers widely supported ES Modules. Today, every major browser supports


The post Pika Dependency Management with Fred Schott appeared first on Software Engineering Daily.

Food delivery apps have changed how the restaurant world operates. After seven years of mobile food delivery, the volume of food ordered through these apps has become so large that entire restaurants can be sustained solely through the order flow that comes in from the apps. This raises the question as to why you even


The post Cloud Kitchen Platform with Ashley Colpaart appeared first on Software Engineering Daily.

Remote engineering work makes some elements of software development harder, and some elements easier. With Slack and email, communication becomes more clear cut. Project management tools lay out the responsibilities and deliverables of each person. GitHub centralizes and defines the roles of developers. On the other hand, remote work subtracts the role of nuanced conversation.


The post Remote Team Management with Ryan Chartrand appeared first on Software Engineering Daily.

Programming languages are dynamically typed or statically typed. In a dynamically typed language, the programmer does not need to declare if a variable is an integer, string, or other type. In a statically typed language, the developer must declare the type of the variable upfront, so that the compiler can take advantage of that information.


The post Sorbet: Typed Ruby with Dmitry Petrashko appeared first on Software Engineering Daily.

Datomic is a database system based on an append-only record keeping system. Datomic users can query the complete history of the database, and Datomic has ACID transactional support. The data within Datomic is stored in an underlying database system such as Cassandra or Postgres. The database is written in Clojure, and was co-authored by the


The post Datomic Architecture with Marshall Thompson appeared first on Software Engineering Daily.

A large cloud provider has high volumes of network traffic moving through data centers throughout the world. These providers manage the infrastructure for thousands of companies, across racks and racks of multitenant servers, and cables that stretch underseas, connecting network packets with their destination. Google Cloud Platform has grown steadily into a wide range of


The post Google Cloud Networking with Lakshmi Sharma appeared first on Software Engineering Daily.

Over the last fifteen years, there has been a massive increase in the number of new software tools. This is true at the infrastructure layer: there are more databases, more cloud providers, and more open-source projects. And it’s also true at a higher level: there are more APIs, project management systems, and productivity tools. ClickUp


The post ClickUp Engineering with Zeb Evans and Alex Yurkowski appeared first on Software Engineering Daily.

Infrastructure-as-code allows developers to use programming languages to define the architecture of their software deployments, including servers, load balancers, and databases.  There have been several generations of infrastructure-as-code tools. Systems such as Chef, Puppet, Salt, and Ansible provided a domain-specific imperative scripting language that became popular along with the early growth of Amazon Web Services.


The post Pulumi: Infrastructure as Code with Joe Duffy appeared first on Software Engineering Daily.

Software investing requires a deep understanding of the market, and an ability to predict what changes might occur in the near future. At the level of core infrastructure, software investing is particularly difficult. Databases, virtualization, and large scale data processing tools are all complicated, highly competitive areas. As the software world has matured, it has


The post Infrastructure Investing with Vivek Saraswat appeared first on Software Engineering Daily.

A high volume of data can contain a high volume of useful information. That fact is well understood by the software world. Unfortunately, it is not a simple process to surface useful information from this high volume of data. A human analyst needs to understand the business, formulate a question, and determine what metrics could


The post Sisu Data with Peter Bailis appeared first on Software Engineering Daily.

Physical places have a large amount of latent data. Pick any location on a map, and think about all of the questions you could ask about that location. What businesses are at that location? How many cars pass through it? What is the soil composition? How much is the land on that location worth? The


The post Location Data with Ryan Fox Squire appeared first on Software Engineering Daily.

Descript is a software product for editing podcasts and video. Descript is a deceptively powerful tool, and its software architecture includes novel usage of transcription APIs, text-to-speech, speech-to-text, and other domain-specific machine learning applications. Some of the most popular podcasts and YouTube channels use Descript as their editing tool because it provides a set of


The post Descript with Andrew Mason appeared first on Software Engineering Daily.

Lyft is a ridesharing company that generates a high volume of data every day.  This data includes ride history, pricing information, mapping, routing, and financial transactions. The data is stored across a variety of different databases, data lakes, and queueing systems, and is processed at scale in order to generate machine learning models, reports, and


The post Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare appeared first on Software Engineering Daily.

Cloud computing caused a fundamental economic shift in how software is built. Before the cloud, businesses needed to buy physical servers in order to operate. There was an up-front cost that often amounted to tens of thousands of dollars required to pay for these servers.  Cloud computing changed the up-front capital expense to an ongoing


The post Cloud Investing with Danel Dayan appeared first on Software Engineering Daily.

GraphQL is a system that allows frontend engineers to make requests across multiple data sources using a simple query format. In GraphQL, a frontend developer does not have to worry about the request logic for individual backend services. The frontend developer only needs to know how to issue GraphQL requests from the client, and these


The post OneGraph: GraphQL Tooling with Sean Grove appeared first on Software Engineering Daily.

A data warehouse serves the purpose of providing low latency queries for high volumes of data. A data warehouse is often part of a data pipeline, which moves data through different areas of infrastructure in order to build applications such as machine learning models, dashboards, and reports. Modern data pipelines are often associated with the


The post DBT: Data Build Tool with Tristan Handy appeared first on Software Engineering Daily.

ReactJS developers have lots of options for building their applications, and those options are not easy to work through. State management, concurrency, networking, and testing all have elements of complexity and a wide range of available tools. Take a look at any specific area of JavaScript application development, and you can find highly varied opinions.


The post React Best Practices with Kent Dodds appeared first on Software Engineering Daily.

JavaScript fatigue. This phrase has been used to describe the confusion and exhaustion around the volume of different tools required to be productive as a JavaScript developer. Frameworks, package managers, typing systems, state management, GraphQL, and deployment systems–there are so many decisions to make. In addition to the present-day tooling choices, a JavaScript developer needs


The post React Stack with Tejas Kumar appeared first on Software Engineering Daily.

Full-stack JavaScript applications have been possible since the creation of NodeJS in 2009. Since then, the best practices for building and deploying these applications have steadily evolved with the technology.  ReactJS created consolidation around the view layer. The emergence of AWS Lambda created a new paradigm for backend execution. Serverless tools such as DynamoDB offer


The post JavaScript Deployments with Brian LeRoux appeared first on Software Engineering Daily.

ReactJS began to standardize frontend web development around 2015. The core ideas around one-way data binding, JSX, and components caused many developers to embrace React with open arms. There has been a large number of educators that have emerged to help train developers wanting to learn React. A new developer learning React has numerous questions


The post React Fundamentals with Ryan Florence appeared first on Software Engineering Daily.

When ReactJS became popular, frontend web development became easier. But React is just a view layer. Developers who came to React expecting a full web development framework like Ruby on Rails or Django were required to put together a set of tools to satisfy that purpose. A full-stack JavaScript framework has numerous requirements. How does


The post NextJS with Guillermo Rauch appeared first on Software Engineering Daily.

Slack is a messaging application with millions of users. The desktop application is an Electron app, which is effectively a web browser dedicated to running Slack. This frontend is built with ReactJS and other JavaScript code, and the application is incredibly smooth and reliable, despite its complexity. When a user boots up Slack, the application


The post Slack Frontend Architecture with Anuj Nair appeared first on Software Engineering Daily.

Low code tools can be used to build an increasing number of applications. Knowledge workers within a large corporation can use low code tools to augment their usage of spreadsheets. Entrepreneurs can use low code tools to start businesses even without knowing how to code. Modern low code tools have benefited from steady improvements in


The post Makerpad: Low Code Tools with Ben Tossell appeared first on Software Engineering Daily.

Every company has a large number of routine data workflows. These data workflows involve spreadsheets, CSV files, and tedious manual work to be done by a knowledge worker.  For example, data might need to be taken from Salesforce, filtered for new customers, and piped into Mailchimp. Or perhaps you need to sort all your customers


The post Parabola: No-Code Data Workflows with Alex Yaseen appeared first on Software Engineering Daily.

Cryptocurrencies today serve two purposes: store of value and speculation.  The application infrastructure that has been built around cryptocurrency is mostly to support these use cases. At some point in the future, perhaps cryptocurrencies can be used as a global medium of exchange that is accepted at the grocery store. Perhaps we will use the


The post Decentralized Finance with Tom Schmidt appeared first on Software Engineering Daily.

At Airbnb, infrastructure management is standardized across the organization. Platform engineering teams build tools that allow the other teams throughout the organization to work more effectively. A platform engineering team handles problems such as continuous integration, observability, and service discovery. Other teams throughout a company use the tools that a platform engineering team builds. For


The post Infrastructure Management with Joey Parsons appeared first on Software Engineering Daily.

A gig economy application generates lots of notifications.  There is SMS, mobile phone updates, emails, and native application updates. If you order a ride from Uber, you might receive a text message and a push notification at the same time. If an app overloads the user with notifications, the user might end up annoyed and


The post Courier with Troy Goode appeared first on Software Engineering Daily.

In a modern data platform, distributed streaming systems are used to read data coming off of an application in real-time. There are a wide variety of streaming systems, including Kafka Streams, Apache Samza, Apache Flink, Spark Streaming, and more.  When Eric Anderson joined the show back in 2016, he was working at Google on Google


The post Data Infrastructure Investing with Eric Anderson appeared first on Software Engineering Daily.

Distributed stream processing frameworks are used to rapidly ingest and aggregate large volumes of incoming data. These frameworks often require the application developer to write imperative logic describing how that data should be processed.  For example, a high volume of clickstream data that is getting buffered to Kafka needs to have a stream processing system


The post Materialize: Streaming SQL on Timely Data with Arjun Narayan and Frank McSherry appeared first on Software Engineering Daily.

A cloud provider gives developers access to virtualized server infrastructure. When a developer rents this infrastructure via an API call, a virtual server is instantiated on physical machines. That virtual server needs to be made addressable through the allocation of an IP address to make it reachable from the open Internet. When the virtual server


The post Go Networking with Sneha Inguva appeared first on Software Engineering Daily.

A data pipeline is a series of steps that takes large data sets and creates usable results from them. At the beginning of a data pipeline, a data set might be pulled from a database, a distributed file system, or a Kafka topic. Throughout a data pipeline, different data sets are joined, filtered, and statistically


The post Great Expectations: Data Pipeline Testing with Abe Gong appeared first on Software Engineering Daily.

A data warehouse provides low latency access to large volumes of data.  A data warehouse is a crucial piece of infrastructure for a large company, because it can be used to answer complex questions involving a large number of data points. But a data warehouse usually cannot hold all of a company’s data at any


The post Data Warehouse ETL with Matthew Scullion appeared first on Software Engineering Daily.

Machine learning applications are widely deployed across the software industry.  Most of these applications used supervised learning, a process in which labeled data sets are used to find correlations between the labels and the trends in that underlying data. But supervised learning is only one application of machine learning. Another broad set of machine learning


The post Anyscale with Ion Stoica appeared first on Software Engineering Daily.

Distributed stream processing systems are used to read large volumes of data and perform operations across those data streams.  These stream processing systems often build off of the MapReduce algorithm for collecting and aggregating large volumes of data, but instead of processing a calculation over a single large batch of data, they process data on


The post Flink and BEAM Stream Processing with Maximilian Michels appeared first on Software Engineering Daily.

Large companies generate large volumes of data. This data gets dumped into a data lake for long-term storage, then pulled into memory for processing and analysis. Once it is in memory, it is often read into a dashboard, which presents a human with a visualization of the data.  The end-user who is consuming this data


The post Druid Analytics with Jad Nauous appeared first on Software Engineering Daily.

Data infrastructure has been transformed over the last fifteen years.  The open source Hadoop project led to the creation of multiple companies based around commercializing the MapReduce algorithm and Hadoop distributed file system. Cheap cloud storage popularized the usage of data lakes. Cheap cloud servers led to wide experimentation for data tools. Apache Spark emerged


The post The Data Exchange with Ben Lorica appeared first on Software Engineering Daily.

A data platform contains all of the data that a company has accumulated over the years. Across a data platform, there is a multitude of data sources: databases, a data lake, data warehouses, a distributed queue like Kafka, and external data sources like Salesforce and Zendesk. A user of the data platform often has a


The post Presto with Justin Borgman appeared first on Software Engineering Daily.

Nubank is a popular bank that is based in Brazil. Nubank has more than 20 million customers, and has accumulated a high volume of data over the six years since it was started. Mobile computing and cloud computing have given rise to “challenger banks” that operate more like software companies. When a software company reaches


The post Nubank Data Engineering with Sujith Nair appeared first on Software Engineering Daily.

The Changelog is a podcast about the world of open source. As open source has become closely tied with the entire software development lifecycle, The Changelog has expanded its coverage to the broader software industry. Since starting the podcast ten years ago, Adam Stacoviak and Jerod Santo have become full-time podcasters, and they have started


The post Changelog Podcasting with Adam Stacoviak and Jerod Santo appeared first on Software Engineering Daily.

Animations can be used to create games, app tutorials, and user interface components. Animations can be seen in messaging apps, where animated reactions can convey rich feelings over a text interface. Loading screens can become less boring through animation, and voice assistant products can feel more alive through animation. But we still don’t see much


The post Rive: Animation Tooling with Guido and Luigi Rosso appeared first on Software Engineering Daily.

Robotics has changed modern agriculture. Autonomous systems are powering the tractors, cotton pickers, and corn cutters that yield plants at industrial scale. John Deere is a company that has been making farm equipment for 183 years. Over that period, the planting and harvesting process has become increasingly mechanized, and John Deere has been at the


The post John Deere: Farm Software with Ryan Bergman appeared first on Software Engineering Daily.

Venture capital investing requires an understanding of market dynamics, technology, and finance.  There is also an element of human nature. Consumer trends can make or break the viability of a new product. And early stage venture investing is always a bet on a small team or individual founder. Early stage investments are usually into companies


The post Venture Stories with Erik Torenberg appeared first on Software Engineering Daily.

Stock trading takes place across a variety of software platforms. Etrade and Schwab have allowed individual traders to buy securities for decades. Robinhood built a business around a similar model, but also removed the commission. Wealthfront and Betterment provide “roboadvisor” services that abstract away the underlying securities and focus on managing a risk profile. Each


The post Alpaca: Stock Trading API with Yoshi Yokokawa appeared first on Software Engineering Daily.

Large software companies have lots of users, and the activity from those users results in high volumes of traffic. These companies also have a large surface area across the enterprise.  There are hundreds of services and databases that are fulfilling user requests. As these requests enter the infrastructure of the enterprise, the requests travel through


The post Cloud Log Analysis with Jack Naglieri appeared first on Software Engineering Daily.

Distributed systems are required to run most modern enterprise software. Application services need multiple instances for scalability and failover. Large databases are sharded onto multiple nodes. Logging services, streaming frameworks, and continuous integration tools all require the orchestration of more than one server. Deploying a distributed system has historically been difficult because the nodes of


The post Replicated Software Delivery with Grant Miller and Marc Campbell appeared first on Software Engineering Daily.

Chat systems have been a part of software development for decades. Older systems like Pidgin and Yammer were surpassed by newer systems like HipChat. And when Slack was created, it quickly became a part of most software companies. But Slack does not fulfill the needs of every company. Mattermost is an open-source chat system. Mattermost


The post Mattermost with Ian Tien appeared first on Software Engineering Daily.

The word “DevOps” has a different definition depending on who you ask.  For some people, it is about the process of managing and releasing code. It can involve container management and server orchestration. It can involve infrastructure-as-code, and safer configuration management. In addition to a set of technologies, DevOps can be seen as a management


The post GitLab Strategy with Sid Sibrandij appeared first on Software Engineering Daily.

The ridesharing infrastructure of Lyft has a high volume of traffic that is mostly handled by servers on AWS. When Vicki Cheung joined Lyft in 2018, the company was managing containers with an internally built container scheduler. One of her primary goals at the company was to move Lyft to Kubernetes. In today’s episode, Vicki


The post Lyft Kubernetes with Vicki Cheung appeared first on Software Engineering Daily.

If the Internet was reimagined with the software and hardware infrastructure we have today, what would it look like? That is the question that DFINITY is working on answering. DFINITY’s goal is to build a decentralized, secure Internet computer. DFINITY takes concepts from the cryptocurrency world, but it is focused on computation, not financial products.


The post DFINITY: The Internet Computer with Dominic Williams appeared first on Software Engineering Daily.

Webflow is a visual programming tool used by designers, developers, and other technical users. Webflow is a leader in the “low code” or “no code” category of software tools that has become prominent in the last few years.  Webflow has been years in the making. In a previous show with Webflow CEO Vlad Magdalin, he


The post Webflow Engineering with Bryant Chou appeared first on Software Engineering Daily.

Software has changed the way the world functions. The rapid pace of change has made it difficult to know how to navigate the new world. Knowledge workers who want to keep advancing in their careers develop a strategy of continuous learning in order to adapt to these changes. O’Reilly Media has existed for almost 40


The post Software Media with Tim O’Reilly appeared first on Software Engineering Daily.

GraphQL has become a core piece of infrastructure for many software applications. GraphQL is used to make requests that are structured as GraphQL queries and responded to through a GraphQL server. The GraphQL server processes the query and fetches the response from the necessary databases, APIs, and backend services. Around 2016, when GraphQL was becoming


The post Apollo GraphQL with Geoff Schmidt appeared first on Software Engineering Daily.

The JavaScript ecosystem stretches across frontend, backend, and middleware. There are newer tools such as GraphQL, Gatsby, and WebAssembly. There are frameworks like React, Vue, and Angular. There is complex data handling with streams, caches, and TensorFlow.js. JavaScript is unlike any other ecosystem, because a single language can be used to construct every part of


The post JS Party with Kevin Ball appeared first on Software Engineering Daily.

Cloud infrastructure is usually consumed in the form of virtual machines or containers. These VMs or containers are running on a physical host machine that is also running other VMs and containers. This is called multitenancy. Servers across cloud providers such as AWS have a high utilization because there are multiple virtual instances running on


The post Packet: Baremetal Infrastructure with Zachary Smith and Nathan Goulding appeared first on Software Engineering Daily.

Edge computing is the usage of servers that are geographically close to the client device. The first common use case for edge computing was CDNs: content-delivery networks. A content delivery network placed media files such as images and videos on multiple servers throughout the world. These are big files, and they take lots of bandwidth


The post Edge Computing Platform with Jaromir Coufal appeared first on Software Engineering Daily.

Every large company generates large amounts of data. Data engineering is the process of storing, transforming, and leveraging that data. Data infrastructure companies provide tools and platforms for performing data engineering. The last fifteen years has seen a rise in modern data management companies built in a time of decreasing storage costs, an increased volume


The post Data Infrastructure Go-To-Market with Sean Knapp appeared first on Software Engineering Daily.

Slack is a messaging platform for organizations. Since its creation in 2013, Slack has quickly become a core piece of technology used by a wide variety of technology companies, groups, and small teams.  The messages that are sent on Slack are generated at a very high volume, and are extremely sensitive. These messages must be


The post Slack Data Platform with Josh Wills appeared first on Software Engineering Daily.

NoSQL databases provide an interface for storing and accessing data that allows the user to work with data in an “unstructured” fashion. SQL databases require the data in the database to be “normalized,” meaning that each object in the entire database has an entry (or a null value) for each field. One advantage of NoSQL


The post NoSQL Optimization with Rick Houlihan appeared first on Software Engineering Daily.

Amazon EC2 (Elastic Compute Cloud) is a virtualized server product that provides the user with scalable compute infrastructure. EC2 was created in 2006 as one of the first three AWS services along with S3 and Simple Queueing Service. Since then, EC2 has provided the core server infrastructure for many of the companies that have been


The post Amazon EC2 with Dave Brown appeared first on Software Engineering Daily.

Amazon’s container offerings include ECS (Elastic Container Service), EKS (Elastic Kubernetes Service), and Fargate. Through these different offerings, Amazon provides a variety of ways that a user can manage Kubernetes clusters and standalone container instances. The choice of which containerization system to choose depends on the needs of the user, and the tradeoffs they want


The post Amazon Kubernetes with Abby Fuller appeared first on Software Engineering Daily.

When the Kubernetes project was started, Amazon Web Services was the dominant cloud provider. Most of the code that runs AWS is closed source, which prevents an open ecosystem from developing around AWS. Developers who deploy their application onto AWS are opting into a closed, controlled ecosystem, which has both costs and benefits. The software


The post Kubernetes Progress with Kelsey Hightower appeared first on Software Engineering Daily.

freeCodeCamp was started five years ago with the goal of providing free coding education to anyone on the Internet. freeCodeCamp has become the best place to begin learning how to write software. There are many other places that a software engineer should visit on their educational journey, but freeCodeCamp is the best place to start,


The post freeCodeCamp with Quincy Larson appeared first on Software Engineering Daily.

The software category known as “no-code” describes a set of tools that can be used to build software without writing large amounts of code in a programming language. No-code tools use visual interfaces such as spreadsheets and web based drag-and-drop systems. In previous shows, we have covered some of the prominent no-code products such as


The post No Code with Shawn Wang appeared first on Software Engineering Daily.

Roblox is a gaming platform with a large ecosystem of players, creators, game designers, and entrepreneurs. The world of Roblox is a three-dimensional environment where characters and objects interact through a physics engine. Roblox is multiplayer, and users can interact with each other over the Internet. Roblox is not one single game—it is a system


The post Roblox Engineering with Claus Moberg appeared first on Software Engineering Daily.

Cruise is a company that is building a fully automated self-driving car service. The infrastructure of a self-driving car platform presents a large number of new engineering problems. Self-driving cars collect vast quantities of data as they are driving around the city. This data needs to be transferred from the cars onto cloud servers. The


The post Kubernetes at Cruise with Karl Isenberg appeared first on Software Engineering Daily.

The software supply chain includes cloud infrastructure, on-prem proprietary solutions, APIs, programming languages, networking products, and open source software.  Each of these software categories has its own security vulnerabilities, and each category has tools that can help protect a company from attackers that are trying to exploit known vulnerabilities. As open source software has grown


The post Snyk: Open Source Security with Guy Podjarny appeared first on Software Engineering Daily.

GitLab is a company that builds an open source platform for managing git repositories. GitLab was started in 2012, and has grown to have a large enterprise business with additional products such as continuous integration and security tooling. GitLab is also known for being a large, entirely remote workforce. GitLab does not have any offices,


The post GitLab Engineering with Marin Jankovski appeared first on Software Engineering Daily.

Automation has the potential to eliminate rote jobs such as call center workers and truck drivers. The downstream effects of automation also leads to new jobs, such as data labeling and robot operations. The net effect of modern automation technology is unclear, but it is likely to cause some disruption in the job market. Universal


The post Basic Income with Floyd Marinescu appeared first on Software Engineering Daily.

Logging provides raw data that can be abstracted into higher level information. Logs are generated at every layer of infrastructure: physical host, virtual machine, container, pod, and Kubernetes cluster. Logs are generated by network proxies, edge servers, and API requests. There is far too much logging information to be read by humans.  Log messages need


The post Continuous Intelligence with Kalyan Ramanathan appeared first on Software Engineering Daily.

Offices have historically been the place where most knowledge work takes place. An office is a central meeting point for everyone in an organization. Offices allow for high bandwidth, in-person communication. Employees have access to shared resources, such as food, tables, and quiet working space. Offices provide a means of encouraging a common culture within


The post Remote Work with Philip Thomas appeared first on Software Engineering Daily.

Machine learning algorithms have existed for decades. But in the last ten years, several advancements in software and hardware have caused dramatic growth in the viability of applications based on machine learning. Smartphones generate large quantities of data about how humans move through the world. Software-as-a-service companies generate data about how these humans interact with


The post Practical AI with Chris Benson appeared first on Software Engineering Daily.

The container orchestration wars ended in 2016 with Kubernetes being the most popular open source tool for deploying and managing infrastructure. Since that time, most large enterprises have been implementing a “platform strategy” based around Kubernetes. A platform strategy is a plan for creating a consistent experience for software engineers working throughout an enterprise. At


The post Linkerd Market Strategy with William Morgan appeared first on Software Engineering Daily.

Kubernetes has created a widespread system for deploying and managing infrastructure. As Kubernetes has been increasingly adopted, companies are thinking about how to leverage that common layer of infrastructure. With the common infrastructure abstraction of Kubernetes, it becomes easier to adopt other abstractions that are uniform across the entire company.  This has created a market


The post Istio Market Strategy with Zack Butcher appeared first on Software Engineering Daily.

A cloud provider gives a developer low-cost compute infrastructure on-demand.  Cloud providers can be divided up into two categories: Layer 1 cloud providers and Layer 2 cloud providers. A Layer 1 cloud provider such as Amazon Web Services owns server hardware and sells compute infrastructure as a commodity. A Layer 2 cloud provider purchases compute


The post Heroku Infrastructure with Mark Turner appeared first on Software Engineering Daily.

Java has been popular since the 90s, when it started to be used as a programming language for enterprises.  Today, Java is still widely deployed, but the infrastructure environment is dramatically different. Java is often deployed to containers in the cloud. If those containers can share resources, then those containers can share the same underlying


The post Java 13 with Georges Saab appeared first on Software Engineering Daily.

Relational databases provide durable transactional systems for storing data. The relational model has existed for decades, but the requirements for a relational database have changed. Modern applications have requirements for high volumes of data that do not fit onto a single machine. When a database gets too big to fit on a single machine, that


The post Distributed SQL with Karthik Ranganathan and Sidharth Choudhury appeared first on Software Engineering Daily.

Originally published July 14, 2016 EconTalk is a weekly economics podcast that has been going for a decade. On EconTalk, Russ Roberts brings on writers, intellectuals, and entrepreneurs for engaging conversations about the world as seen through the lens of economics. Russ Roberts is today’s guest, and it is a treat because I have been


The post Economics of Software with Russ Roberts Holiday Repeat appeared first on Software Engineering Daily.

Originally published May 24, 2018 When a user takes a ride on Uber, the app on the user’s phone is communicating with Uber’s backend infrastructure, which is writing to a database that maintains the state of that user’s activity. This database is known as a transactional database or “OLTP” (online transaction processing). Every active user


The post Uber’s Data Platform with Zhenxiao Luo Holiday Repeat appeared first on Software Engineering Daily.

Originally published January 4, 2019 When a user makes a request to product like The New York Times, that request hits an API gateway. An API gateway is the entry point for an external request. An API gateway serves several purposes: authentication, security, routing, load balancing, and logging. API gateways have grown in popularity as


The post Kong API Platform with Marco Palladino Holiday Repeat appeared first on Software Engineering Daily.

Originally published June 7, 2018 Moore’s Law states that the number of transistors in a dense integrated circuit doubles about every two years. Moore’s Law is less like a “law” and more like an observation or a prediction. Moore’s Law is ending. We can no longer fit an increasing amount of transistors in the same


The post Future of Computing with John Hennessy Holiday Repeat appeared first on Software Engineering Daily.

Originally published November 4, 2016 Indie Hackers is a website that profiles independent developers who have made profitable software projects, usually without raising any money. These projects make anywhere from a few hundred dollars a month to more than $100,000 as in the case with park.io, one of the services profiled by Indie Hackers. Courtland


The post Indie Hackers with Courtland Allen Holiday Repeat appeared first on Software Engineering Daily.

New software abstractions always take advantage of the abstractions that have been built before. Software libraries allow us to import code that sits on the same host as a new program. Open source software let us copy and paste existing code, or clone entire repositories. Cloud providers offer hosted tools and APIs that we can


The post Cloud Dependencies with Mya Pitzeruse appeared first on Software Engineering Daily.

Web applications are used on a wide variety of platforms.  On each of these platforms the web app needs to load properly and allow the user to navigate the website and interact with all of the user flows, such as sign-up, login, and the various read and write operations that make up the functionality of


The post Web Application Testing with Gabriel-James Safar appeared first on Software Engineering Daily.

HTTP is a protocol that allows browsers and web applications to communicate across the Internet. Everyone knows that HTTP is doing some important work, because “HTTP” is at the beginning of most URLs that you enter into your browser. You might be familiar with the request/response model, and HTTP request methods such as GET, PUT,


The post HTTP with Julia Evans appeared first on Software Engineering Daily.

Key Values is a platform where companies are profiled with descriptions of their company values. These profiles describe features such as work-life balance, company culture, daily routines, and strategy. Lynne Tye created Key Values with the goal of building a small business that would make money through connecting job seekers to companies with a culture


The post Indie Hack or Venture Back with Lynne Tye appeared first on Software Engineering Daily.

The vision of code-free programming has existed for decades. Software engineers have always dreamed of empowering non-technical users with the same creative tools that programmers have access to.  For many years, the underlying technology of the web was not powerful enough to make this dream a reality. Platforms such as WordPress, Squarespace, and Wix have


The post Bubble: Code-Free Programming with Emmanuel Straschnov and Joshua Haas appeared first on Software Engineering Daily.

“Serverless” is an execution model where applications are scheduled and deployed to servers that are not directly managed by the application developer.  In serverless execution, an application only loads and operates when a user actually needs to get a response from that application. This saves on resources, because many applications do not need to run


The post Cloudflare Serverless with Zack Bloom appeared first on Software Engineering Daily.

Data engineering is difficult.  Companies want to be able to maximize the value they get from their large data sets, but there are so many steps required for deriving that value that most companies feel like they are always far behind the ideal state of where they could be.  The cloud makes it cheap to


The post Dagster with Nick Schrock appeared first on Software Engineering Daily.

Java programs run in a different environment than they did ten years ago. Modern infrastructure runs on containers sitting in a Kubernetes cluster. The optimal configuration for a Java program in that context is different than it was for an environment dominated by virtual machines and bare metal. When you are co-scheduling your services with


The post GraalVM Quarkus: Java Acceleration with Guillaume Smet and Emmanuel Bernard appeared first on Software Engineering Daily.

Crash monitoring emerged as a software category over the last decade. Crash monitoring software allows developers to understand when their applications are crashing on client devices. For example, we have an app for Software Engineering Daily that people download on Android or iOS. Users download the app to their smartphone. When the user is playing


The post Bugsnag Business with James Smith appeared first on Software Engineering Daily.

Software bugs cause unexpected problems at every company.  Some problems are small. A website goes down in the middle of the night, and the outage triggers a phone call to an engineer who has to wake up and fix the problem. Other problems can be significantly larger. When a major problem occurs, it can cause


The post Incident Response Machine Learning with Chris Riley appeared first on Software Engineering Daily.

Frontend software development has become as complex as backend development. There was a time when frontend web development was simple. There was a small number of JavaScript frameworks and templating systems. Your CSS was simple configuration for the colors on your webpage. Today, there is a giant ecosystem of frontend tools, APIs, and middleware delivering


The post Gatsby with Kyle Mathews and Sam Bhagwat appeared first on Software Engineering Daily.

Drones will deliver food to us. Drones will be able to extinguish fires. Drones will be used to relay Internet signals and make the world more connected.  These all sound like great ideas, so why aren’t there more drones in the sky today? There are many answers to that question, some of which relate to


The post Drones with Chris Anderson appeared first on Software Engineering Daily.

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data. Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure. Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed,


The post LinkedIn Data Engineering with Kapil Surlaker appeared first on Software Engineering Daily.

Every company has a software supply chain. A company builds its products from custom code, paid APIs, paid proprietary binaries, and open source software libraries. As the types of software available have increased, the management of the software supply chain has become complex.  Large software companies have always needed to ensure the security of their


The post FOSSA: Open Source Management with Kevin Wang appeared first on Software Engineering Daily.

Machine learning is widely understood by the software community. But it is still hard to build a company around machine learning, because there is not easy access to large, unique data sets. Scale is a platform for training and validating data that is used for machine learning. Most machine learning models are built with supervised


The post Scale with Alexandr Wang appeared first on Software Engineering Daily.

Photo credit: Elisabeth Fall Ben Horowitz started Loudcloud with Marc Andreessen in 1999. He ran the company for eight years and chronicled his experience in his first book The Hard Thing About Hard Things. In his time running Loudcloud, the dotcom bubble burst, but Loudcloud needed cash so badly that he took the company public


The post Leadership with Ben Horowitz appeared first on Software Engineering Daily.

Facebook leadership has a significant amount of engineers in its ranks, and engineers understand how to create an environment that appeals to other engineers.  Engineers do not like working on projects that they are not interested in, so Facebook optimizes for matching engineers to enjoyable work. Engineers do not like taking orders from managers, so


The post Facebook Leadership with Arturo Bejar appeared first on Software Engineering Daily.

Chaos Monkeys is a book about Silicon Valley startups and Facebook. It is one of the most accurate books written about the modern technology industry, and captures both the negatives and the positives of software companies. Antonio Garcia Martinez is the author of Chaos Monkeys. He wrote the book after going through a gauntlet of


The post Facebook Fallout with Antonio Garcia Martinez appeared first on Software Engineering Daily.

Facebook engineering is unique. Software is built at Facebook in a way that is distinctly different than any other company. In our series of shows about Facebook engineering, we have mostly covered the positive side of Facebook’s practices. In today’s show, we explore the downsides.  Facebook moves fast. Engineers within the company must move fast,


The post Facebook Reflections with Pete Hunt and Nick Schrock appeared first on Software Engineering Daily.

Peter Deng has worked on most of Facebook’s major products: Newsfeed, Instagram, Oculus, and Messenger. These different products have different requirements, but are all part of the same ethos of connecting people through social networks.  Facebook is a consumer product company that is powered by a strong engineering workforce. The relationship between product managers and


The post Facebook Products with Peter Deng appeared first on Software Engineering Daily.

Jeff Rothschild was one of the earliest engineers to join Facebook. In the 1990s, Jeff had co-founded Veritas Software and helped it to its IPO in 2004. After Veritas, Jeff worked on several other of his own companies. He was working with Accel Partners on investments when he started to learn about Facebook. Accel was


The post Facebook Entrepreneurship with Jeff Rothschild appeared first on Software Engineering Daily.

In 2009, the Berkeley AMPLab was a center of innovation.  Three projects from AMPLab have turned into successful open source projects and companies: Spark, Mesos, and Alluxio. Haoyuan Li was the creator of Alluxio, and he returns to the show to discuss his journey taking Alluxio from a research project to a company that has


The post Alluxio: Data Orchestration with Haoyuan Li appeared first on Software Engineering Daily.

Redis is an in-memory database that persists to disk. Redis is commonly used as an object cache for web applications. Applications are composed of caches and databases. A cache typically stores the data in memory, and a database typically stores the data on disk. Memory has significantly faster access times, but is more expensive and


The post Redis with Alvin Richards appeared first on Software Engineering Daily.

LinkedIn is a social network with petabytes of data.  In order to store that data, LinkedIn distributes and replicates that data across a large cluster of machines running the Hadoop Distributed File System. In order to run calculations across its large data set, LinkedIn needs to split the computation up using MapReduce-style jobs. LinkedIn has


The post LinkedIn Data Platform with Carl Steinbach appeared first on Software Engineering Daily.

Cryptocurrencies are a fundamental computer science invention.  Cryptocurrencies crashed in 2018 but the technology remains as promising as ever. Bitcoin is a decentralized currency, and a plausible end state that is implied by Bitcoin’s current trajectory is a permissionless, decentralized financial system. This idea of decentralized finance or “DeFi” begs numerous questions: who will build


The post Crypto Businesses with Haseeb Qureshi appeared first on Software Engineering Daily.

Dark Lang is a programming language that is tightly integrated with the cloud.  Dark takes an opinionated approach that most developers are going to want to run their applications in the cloud, and this perspective influences how Dark looks at deployments, IDEs, exception handling, and other aspects of software development. Paul Biggar is the founder


The post Dark Lang with Ellen Chisa and Paul Biggar appeared first on Software Engineering Daily.

Apache Kafka was created at LinkedIn. Kafka was open sourced in 2011, when the company was eight years old. By that time, LinkedIn had developed a social network with millions of users. LinkedIn’s engineering team was building a range of externally facing products and internal tools, and many of these tools required a high-throughput system


The post LinkedIn Kafka with Nacho Solis appeared first on Software Engineering Daily.

Modern applications are distributed systems. These applications require an installation mechanism that can run and update the software across multiple nodes.  When a SaaS company starts to work with large enterprise customers, that company needs to figure out a way to deliver their product to the enterprise. This requires the SaaS company to deploy the


The post Gravity: Distributed Application Delivery with Ev Kontsevoy appeared first on Software Engineering Daily.

Databases go offline. Services fail to scale up. Deployment errors can cause an application backend to get DDoS’d. When an event happens that prevents your company from operating as expected, it is known as an incident. Software teams respond to an incident by issuing a fix. Sometimes that fix returns the software to its ideal


The post Incident Reproduction with Tammy Butow appeared first on Software Engineering Daily.

Diffbot is a knowledge graph that allows developers to interface with the unstructured web as if it was a structured database. In today’s show, Diffbot CEO Mike Tung returns for a second discussion about how he has built Diffbot and how Diffbot is used. The web has many different entities. Web pages, topics, people, stories,


The post Diffbot Infrastructure with Mike Tung appeared first on Software Engineering Daily.

Render is a cloud provider built on top of Amazon Web Services and Google Cloud. Render uses the compute abstractions provided by the major cloud providers to build a second layer cloud provider with the goal of providing a better user experience. Anurag Goel is the founder of Render, and he returns to the show


The post How To Build A Cloud Provider with Anurag Goel appeared first on Software Engineering Daily.

We are republishing a podcast from the freeCodeCamp Podcast as a weekend episode. Jeff Meyerson is the creator and host of the Software Engineering Daily podcast. Jeff grew up in Texas. He spent most of his childhood playing competitive strategy games like Magic: the Gathering. This lead him to making a lot of money –


The post freeCodeCamp’s Quincy Larson interviews Jeff Meyerson appeared first on Software Engineering Daily.

Webflow is a platform for building applications without programming. Software engineering has barely been around for 30 years. Over that period of time, there have been many attempts to create a platform that allows for the creation of software without writing a line of code. Most of these systems have not been able to fulfill


The post Webflow: No-Code with Vlad Magdolin appeared first on Software Engineering Daily.

Data visualization is the presentation of data in a way that emphasizes certain qualities about that data. Data visualization can be used to prove a specific point, or it can be used as a depiction of a data set to be explored. Data visualization is used in consumer software products as well as backend engineering


The post Data Visualization with Sherman Wood and Chad Lumley appeared first on Software Engineering Daily.

The ability to create software is a superpower. But software education is not evenly distributed. Ryan Carson started Treehouse to provide a high quality education system for anyone to learn how to build software. On a previous episode, Ryan and I discussed the field of programming education. Ryan returns to the show for a conversation


The post Treehouse: Business and Education with Ryan Carson appeared first on Software Engineering Daily.

Video surveillance impacts human lives every day.  On most days, we do not feel the impact of video surveillance. But the effects of video surveillance have tremendous potential. It can be used to solve crimes and find missing children. It can be used to intimidate journalists and empower dictators. Like any piece of technology, video


The post Traces: Video Recognition with Veronica Yurchuk and Kostyantyn Shysh appeared first on Software Engineering Daily.

Data privacy policies have changed how software organizations need to operate. As consumer preferences have shifted in favor of strong privacy, software companies are having to examine their policies around data collection and retention. Many software companies were started in a time with different norms around data.  Building a new application that is compliant with


The post GDPR in Practice with Joshua Prismon appeared first on Software Engineering Daily.

Indie Hackers is a platform for independent software businesses to discuss strategy and find inspiration. Courtland Allen founded Indie Hackers with the goal of sharing the stories of these businesses, and the company has become a thriving community of entrepreneurs, engineers, and creators. Business is a creative medium. The definition of a successful business is


The post Indie Hackers (3 Years Later) with Courtland Allen appeared first on Software Engineering Daily.

Scott Belsky founded Behance in 2006. Behance is a social platform where designers and creators share their work. Scott was motivated to start Behance due to his desire to combine his love for creativity with his desire to create a business. After 6 years of work, Behance was acquired by Adobe for more than $150


The post The Messy Middle with Scott Belsky appeared first on Software Engineering Daily.

Large companies have multiple databases, multiple data formats, and multiple applications that need to use the data. Every data engineer needs to move data between these different components of a system. Moving data between different parts of a system is often called “ETL”, an acronym for “Extract, Transform, Load.” Data engineers spend much of their


The post Fivetran: Data Connectors with George Fraser appeared first on Software Engineering Daily.

The development of self-driving cars is one of the biggest technological changes that is under way. Across the world, thousands of engineers are working on developing self-driving cars. Although it still seems far away, self-driving cars are starting to feel like an inevitability. This is especially true if you spend much time in downtown San


The post Cruise: Self-Driving Engineering with Mo Elshenawy appeared first on Software Engineering Daily.

Investors often use the term “moat” to describe the durable competitive advantage of a company. When an investor puts money into a company, they are making that investment based on a valuation. That valuation is subjective–it is how much the investor thinks the company is worth. A valuation is determined by the present value of


The post Software Moats with Astasia Myers appeared first on Software Engineering Daily.

Software engineering is a new field. There are theories about how we should be building our systems, but these theories might change over time. The same is true for engineering management.  There are many successful examples of companies scaling with the management hierarchies pioneered by Microsoft and Google, but since everyone knows that those techniques


The post Stripe Infrastructure Management with Uma Chingunde appeared first on Software Engineering Daily.

Ethos Life Insurance is a software company that sells life insurance products. Software is reshaping established industries such as banking, insurance, and manufacturing. In these large, established industries, incumbents are adopting new technology as fast as they can, but the new technology needs to be integrated with the old technology. The slow rate of technology


The post Life Insurance Engineering with Vipul Sharma and Lingke Wang appeared first on Software Engineering Daily.

Isolation is a fundamental concept in computer science. Software workloads are isolated from each other in order to keep resource access cleanly separated.  When programs are properly isolated, it is easier for the programmer to reason about the memory safety of that program. When a program is not properly isolated, it can lead problems such


The post WebAssembly Isolation with Tyler McMullen appeared first on Software Engineering Daily.

Cloud Foundry is a system for managing distributed applications. Cloud Foundry was released in 2011, and has been widely adopted by enterprises that need a platform for deploying and scaling the applications that run within their company. The ecosystem around Cloud Foundry includes systems for continuous delivery, pubsub messaging, and containerization. Abby Kearns is the


The post Cloud Foundry with Abby Kearns appeared first on Software Engineering Daily.

A new software product usually starts with a single database. That database manages the tables for user accounts and basic transactions. When a product becomes popular, the database grows in size. There are more transactions and more users. A company grows around that product, and the company starts to accumulate more data in different sources.


The post Kafka Data Pipelines with Robin Moffatt appeared first on Software Engineering Daily.

A software company needs to get many things right in order to be successful. Having a useful product with solid engineering is only the beginning. ReadMe was started five years ago. The company solved a seemingly simple problem: documentation for software products. If you have worked as a software engineer, you have looked at documentation.


The post ReadMe with Greg Koberger appeared first on Software Engineering Daily.

Open source software is very new. Open source has existed for less than 30-40 years, depending on who you ask. The idea of open source was popularized by Linux, and open source software started to get heavily commercialized in the 1990s. By the early 2000s, open source was used by nearly every large software company.


The post Open Source Ecosystem with Dirk Hohndel appeared first on Software Engineering Daily.

Modern databases consist of multiple servers that host the data in a distributed fashion. Using multiple servers allows a database to be resilient to the failure of any one database node, because copies of the data are shared to other servers. A multi-node setup also lets the database grow beyond the size of data that


The post Distributed Databases with Aly Cabral appeared first on Software Engineering Daily.

Ever since Apache Kafka was open sourced from LinkedIn, it has been used to solve a wide variety of problems in distributed systems and data engineering. Kafka is a distributed messaging queue that is used by developers to publish messages and subscribe to topics with a certain message type. Kafka allows information to flow throughout


The post Kafka Applications with Tim Berglund appeared first on Software Engineering Daily.

A new employee at a software company needs access to a variety of tools. In order to get started working, the employee might need Slack, email, Google Docs, and Amazon Web Services, and all of these require an account with a username and password. Setting up all of these accounts can be time consuming, because


The post Okta Engineering with Hector Aguilar appeared first on Software Engineering Daily.

Amazon Web Services first came out in 2006. It took several years before the software industry realized that cloud computing was a transformative piece of technology. Initially, the common perspective around cloud computing was that it was a useful tool for startups, but would not be a smart option for large, established businesses. Cloud computing


The post Cloud-Native Applications with Cornelia Davis appeared first on Software Engineering Daily.

Python is one of the most popular programming languages in the software world. After working with Python and developing a love for the language, Michael Kennedy started to wonder why there was not a high quality podcast dedicated to covering the community and new technologies of the Python ecosystem. Michael started Talk Python To Me


The post Talking Python with Michael Kennedy appeared first on Software Engineering Daily.

A new software application has simple requirements for a database.  The database needs to be written to and read from. The database fulfills simple needs such as storing user information and providing the application frontend with the necessary data to render a simple webpage of financial transactions or blog posts. As an application becomes successful,


The post MongoDB Data Platform with Andrew Davidson appeared first on Software Engineering Daily.

Spanner is a globally distributed, transactionally consistent database.  Spanner initially emerged as a paper that came out of Google in 2012. Around this time, database scalability was difficult to solve, even for Google. The Spanner paper offered some breakthroughs in distributed systems which allowed Google to take some of the learnings from BigTable’s eventual consistency


The post Google Spanner with Deepti Srivastava appeared first on Software Engineering Daily.

Open source plays a key role in today’s world of technology businesses. Today, the impact of open source seems obvious. From Kubernetes to distributed databases to cloud providers, so much of our software is powered by open source. But it was not always this way.   Bruce Perens was one of the earliest figures in the


The post Open Source Policy with Bruce Perens appeared first on Software Engineering Daily.

Host: Charles Max Wood of JavaScript Jabber Joined by Special Guest: Jeffrey Meyerson Jeffrey Meyerson, founder of FindCollabs and host at Software Engineering Daily joins Charles Max Wood for a discussion about latest trends in the developer world, ways of monetizing podcasts, and finding ads for podcasts. Jeffrey shares how he started to host podcasts and how he became


The post JavaScript Jabber with Jeff Meyerson appeared first on Software Engineering Daily.

The browser has become the central application of the consumer operating system. Every piece of client software, from email to document management, has become usable through the browser. Even modern desktop software such as Slack is built using Electron, a tool for building client applications that essentially run inside of a browser without an address


The post Repl.it: Browser Coding with Amjad Masad appeared first on Software Engineering Daily.

Modern software is built with cloud services, APIs, and other high level tools. Technical software development is moving beyond the realm of writing code. Individuals who do not have a background in computer science or programming can create increasingly complex tools. Higher level APIs include Twilio for managing phone communications, and Stripe for managing financial


The post Monday: Business Management Software with Eran Zinman appeared first on Software Engineering Daily.

Cassandra was initially released in 2008 as a project out of Facebook. Cassandra offered an open source solution to database scalability issues that were being tackled internally by large companies like Amazon, Google and Facebook. 2008 was a golden age of new infrastructure, with systems such as Hadoop and Kafka gaining traction around the same


The post Cassandra Business with Jonathan Ellis appeared first on Software Engineering Daily.

Airlines have always had an emphasis on new technology. Over the years, airlines have needed to develop more and more software. Digital transformation is causing every large company to adopt the tools and practices of software companies, and that includes Delta Airlines. Delta Airlines has existed for more than 90 years. Over that period of


The post DevOps at Delta Air Lines with Jasmine James appeared first on Software Engineering Daily.

Ilya Sukhar founded Parse in 2011 to make cloud services for mobile developers. Parse was a newer kind of cloud provider called “backend-as-a-service”, built to simplify the complexities of Amazon Web Services and the complexities of the mobile ecosystem.  During this time, Facebook was in the middle of its shift toward becoming a mobile application


The post Facebook Parse Acquisition (Part 2) with Ilya Sukhar appeared first on Software Engineering Daily.

Parse was a backend-as-a-service company that started in 2011 with the initial focus of making the cloud easier to use for mobile developers. Parse had several novel engineering challenges. In 2011, it was not easy to build on top of AWS, nor was it easy to build within the young mobile ecosystem. Charity Majors was


The post Facebook Parse Acquisition (Part 1) with Charity Majors appeared first on Software Engineering Daily.

Kent Beck is a legendary figure in the world of software engineering.  Kent was an early advocate of Test-Driven Development (TDD), and popularized the idea of writing unit tests before writing code that would satisfy those unit tests. A unit test isolates and tests a small piece of functionality within a large piece of software.


The post Facebook Engineering Process with Kent Beck appeared first on Software Engineering Daily.

When Chuck Rossi joined Facebook in 2008, he was one of the most experienced release engineers at the company. As he began to explore the engineering practices of the organization, he was surprised, confused, and impressed by the release engineering system that he encountered. Release engineering is the process by which software is released to


The post Facebook Release Engineering with Chuck Rossi appeared first on Software Engineering Daily.

Facebook is a large multiuser application. Scaling Facebook was different than scaling a single-user application such as an ecommerce store or a search engine. A social network is faced with unique infrastructure scalability challenges, as well as subjective questions around user communications, privacy, and content. Pedram Keyani worked at Google before joining Facebook in 2007.


The post Facebook Scaling with Pedram Keyani appeared first on Software Engineering Daily.

Cryptocurrencies are decentralized monetary systems built on open source software. The open source software movement has evolved from the world of Linux, MySQL, and Apache to a thriving ecosystem of commercial enterprises built around open source software. This ecosystem includes projects such as Kubernetes, MongoDB, and ReactJS. It includes large organizations such as Amazon Web


The post Crypto and OSS with Haseeb Qureshi, Joseph Jacks, and Alok Vasudev appeared first on Software Engineering Daily.

When a new employee joins a software company, it is often unclear where that employee should begin. Do they have a mentor? What are they working on? What are the expectations for how fast that employee should be contributing? The early period of employment is often referred to as “onboarding.” During the onboarding period, an


The post Technical Onboarding with Kristen Gallagher appeared first on Software Engineering Daily.

A time series database is optimized for the storage of high volumes of sequential data across time. Time series databases are often organized as columnar data stores that can write large volumes of data quickly. These systems can sometimes tolerate data loss, because the data they are gathering is used for monitoring and other applications


The post Time Series Databases with Rob Skillington appeared first on Software Engineering Daily.

Insurance is an old business. Individuals and businesses have been buying insurance policies for decades. These insurance policies can cost hundreds, thousands, or tens of thousands of dollars per year.   Software is remaking the insurance industry. One way new software can improve the insurance industry is through better brokerage technology. Insurance involves carriers and brokers,


The post Insurance Software with Gordon Wintrob appeared first on Software Engineering Daily.

Open source software has evolved into a thriving, multifaceted ecosystem. Open source encompasses operating systems and databases. Open source embodies both altruism and self-interest. And open source enables thriving businesses from WordPress blogs to hundred billion dollar cloud providers. There is a large set of business models that can be built around a successful open


The post Open Source Business Models with Karthik Ranganathan, Heather Meeker, and Matt Asay appeared first on Software Engineering Daily.

Andreas M. Antonopoulos is the author of several books about cryptocurrency engineering, including Mastering Bitcoin and Mastering Ethereum. In these books, Andreas lays out the systems of economics and computer science that underpin the two most mature decentralized monetary systems. When Andreas originally discovered the Bitcoin whitepaper, he had witnessed the repeated mismanagement of government-backed fiat


The post Bitcoin Ecosystem with Andreas M. Antonopoulos appeared first on Software Engineering Daily.

Software engineers often work as a contractor for some duration of their career. A contractor earns a fixed hourly salary for a defined period of weeks, months, or years. Contract work can be more flexible than full-time work, and often pays more than full-time software engineering, because contract jobs can end at any time, and


The post Moonlight: Software Contracting Platform with Emma Lawler and Philip Thomas appeared first on Software Engineering Daily.

The service mesh abstraction allows for a consistent model for managing and monitoring the different components of a microservices architecture.  In the service mesh pattern, each service is deployed with a sidecar container that contains a service proxy. These sidecars are collectively referred to as the “data plane.” Each sidecar provides the service that it


The post Service Mesh Deployment with Varun Talwar appeared first on Software Engineering Daily.

The PlayStation is a line of game consoles created by Sony. PlayStation devices include the PS2, PS3, PS4, and the PSP mobile system. Tony Godar worked as an engineer in the PlayStation ecosystem for 15 years, and he joins the show to give a retrospective on his time in the console industry. Developing hardware and


The post PlayStation Engineering with Tony Godar appeared first on Software Engineering Daily.

Large software companies have become a target for criticism. Google, Facebook, Amazon and other prominent technology giants find themselves under a kind of scrutiny that is reminiscent of banks in 2008 and oil companies in the early 1900s. Across the planet, there is a growing sentiment that “big tech” has too much power, and that


The post Big Business with Tyler Cowen appeared first on Software Engineering Daily.

In this episode Will Larson, author of An Elegant Puzzle: Systems of Engineering Management, speaks with Uma Chingunde of Stripe and Jeff Meyerson of Software Engineering Daily about engineering management. Will was also featured on SE Daily recently. An Elegant Puzzle is an excellent resource on management techniques and strategies for scaling software organizations. Will’s writing draws from


The post An Elegant Puzzle Virtual Book Club appeared first on Software Engineering Daily.

The a16z Podcast is a show that is produced by Andreessen Horowitz, an investment fund based in Silicon Valley. The a16z Podcast covers topics including software engineering, biology, media, cryptocurrencies and entrepreneurship. A16z is one of the most popular podcasts about technology. Sonal Chokshi is the editor in chief at Andreessen Horowitz and the showrunner


The post a16z Podcasting with Sonal Chokshi appeared first on Software Engineering Daily.

Data-as-a-service businesses offer paid access to data sets. These data sets can be useful for building products or training machine learning models.  There has been steady growth in the tools and practices around processing and storing data. But access to data sets remains a bottleneck for widespread development of machine learning applications in a large


The post Data-As-A-Service with Auren Hoffman appeared first on Software Engineering Daily.

A large sales organization has hundreds of sales people. Each of those sales people manages a set of accounts who they are trying to close sales deals on. Sales people are overseen by managers who ensure that the sales people are performing well. Directors and VPs ensure the scalability and health of the overall sales


The post People.ai: Machine Learning for Sales with Andrey Akselrod appeared first on Software Engineering Daily.

During 2015, Uber was going through rapid scalability. The internal engineering systems were constantly tested by the growing user base. Over the next two years, the number of internal services at Uber would grow from 500 to 2000, and standardizing the monitoring of all these different services became a priority. After working with a variety


The post Jaeger: Distributed Tracing at Uber with Yuri Shkuro appeared first on Software Engineering Daily.

A knowledge base assembles information from a wide variety of sources into a central platform. The most popular knowledge base is Wikipedia, which covers a wide variety of concepts through a system that attempts to remain authoritative and impartial. Other open knowledge platforms include Stack Overflow, which focuses on programming concepts, and Quora, which adds


The post Golden: Intelligent Knowledge Map with Jude Gomila appeared first on Software Engineering Daily.

Coding bootcamps allow anyone to become a programmer at a faster pace than the traditional computer science education system. In the last five years, coding bootcamps have grown rapidly in popularity, with thousands of people gaining the necessary skills to work as a software engineer. Career Karma is a platform that allows individuals to find


The post Career Karma: Coding Bootcamp Platform with Ruben Harris and Artur Meyster appeared first on Software Engineering Daily.

A hackathon is an organized event where participants work together to build a product or tool. Hackathons are about creativity, learning, and exploration. A developer that is participating in a hackathon is often working on something that is outside of their normal day-to-day focus.  Hackathons can provide significant value to the participants. Hackathons have led


The post Hackathons with Jonathan Gottfried appeared first on Software Engineering Daily.

Videos play a key role in the continuing education of a software engineer. Video can capture many different types of content that is useful for engineers: conference talks, tutorial videos, and podcast-style interviews are all popular formats of online video. YouTube has become the predominant source for video content about software engineering. The open nature


The post TechPrimers: Software Engineering YouTube with Ajay Kumar appeared first on Software Engineering Daily.

“Internet of Things” is a term used to describe the increasing connectivity and intelligence of physical objects within our lives.  IoT has manifested within enterprises under the term “Industrial IoT,” as wireless connectivity and machine learning have started to improve devices such as centrifuges, conveyor belts, and factory robotics. In the consumer space, IoT has


The post WebAssembly on IoT with Jonathan Beri appeared first on Software Engineering Daily.

Data engineering involves numerous tools–a data lake, databases, data warehouses, numerous APIs, streaming systems, and microservices. There is no shortage of ways to interact with data and manage data, but many companies are struggling to figure out design patterns and best practices for how to manage data and build data infrastructure. Zhamak Dehgani is a


The post Data Mesh with Zhamak Deghani appeared first on Software Engineering Daily.

Software companies such as Slack, Zoom, and Uber have recently gone public. When a company goes public, they issue a document called an S-1. Within the S-1, there is a wealth of information about the company, providing a detailed story about the company’s business model, economics, and future prospects. The S-1 describes the operating model


The post Software IPOs with Tomasz Tunguz appeared first on Software Engineering Daily.

Envoy is an open source edge and service proxy that was originally developed at Lyft.  Envoy is often deployed as a sidecar application that runs alongside a service and helps that service by providing features such as routing, rate limiting, telemetry, and security policy. Envoy has gained significant traction in the open source community, and


The post Envoy Mobile with Matt Klein appeared first on Software Engineering Daily.

In the developed world, it is easy to take for granted that we grew up with computers. Technology is so pervasive in the United States that we have debates about how early in child development a human should be given a smartphone. Across much of Africa, there is a shortage of access to computers. Children


The post TechLitAfrica: Computer Literacy in Africa with Tyler Cinnamon appeared first on Software Engineering Daily.

The Hadoop ecosystem provided every company with the tools to store and query large amounts of data at a low cost. Since 2005, that ecosystem has expanded with more and more open source applications for data infrastructure.  Data infrastructure includes databases, data lakes, distributed queues, data warehouses, query engines, web applications, on-prem software, closed source,


The post Data Engineering with Tobias Macey appeared first on Software Engineering Daily.

From Africa to India to Asia to South America–computer science and programming are rising in popularity in every emerging market. Each of these markets has regional needs for technology. Just like every culture develops its own food and television, every culture needs different types of applications to run their lives. In Vietnam, the day-to-day life


The post Emerging Markets: Vietnam with Charles Lee appeared first on Software Engineering Daily.

In 2011, Facebook had begun to focus its efforts on mobile development. Mobile phones did not have access to reliable, high bandwidth connections, and the Facebook engineering team needed to find a solution to improve the request latency between mobile clients and the backend Facebook infrastructure. One source of latency was recursive data fetching. If


The post Facebook GraphQL with Lee Byron appeared first on Software Engineering Daily.

Facebook has released open source software projects that have changed the industry. The most impactful projects to date are the React frontend user interface tools: ReactJS and React Native. Before React became popular, there were multiple competing solutions for the dominant frontend JavaScript framework. React became the most prominent because of its invention of JSX,


The post Facebook Open Source Management with Tom Occhino appeared first on Software Engineering Daily.

Facebook generates high volumes of data at a rapid pace. Dhruba Borthakur joined Facebook in 2008 to work on data infrastructure. His early projects at Facebook were around Hadoop, the distributed file system and MapReduce computation platform that laid the foundation for the “big data” movement.  At the time, Facebook was generating as much data


The post Facebook Data Infrastructure with Dhruba Borthakur appeared first on Software Engineering Daily.

Facebook moves fast because of vision, collaboration, and trust. The fast pace of development is enabled by constantly improving infrastructure and a sense of unity throughout the company.  In Facebook’s early days, there was an emphasis on rapidly deploying new code to drive constant improvement and experimentation.  Product quality was maintained by engineers closely checking


The post Facebook Engineering Culture with Raylene Yung appeared first on Software Engineering Daily.

Facebook was built using PHP, a programming language that was used widely in the late 90s and early 2000s. PHP allows developers to get web applications built quickly and easily, although PHP has a reputation for being difficult to scale.  In the early days of Facebook, the company was scaling rapidly on every dimension.  New


The post Facebook PHP with Keith Adams appeared first on Software Engineering Daily.

Data engineering allows a company to take advantage of the large quantities of data that the company has generated. In many companies, new data has been produced rapidly for many years, but the company has not been able to take full advantage of it.  Creating large data sets does not provide immediate value for a


The post Meltano: Data Engineering Lifecycle with Danielle Morrill appeared first on Software Engineering Daily.

Software engineers have a wide variety of media to choose from, including podcasts, blogs, YouTube videos, conferences. The amount of software engineering media that is available is growing and accelerating. Eight years ago, there were not as many options for information about software. Charles Max Wood founded Devchat.tv to create a network of podcasts and


The post Software Media with Charles Max Wood appeared first on Software Engineering Daily.

Software applications running within a host operating system need to be isolated. Isolation prevents security vulnerabilities, such as one application accessing the memory of another. In modern cloud environments, a single physical host might be running multiple virtual machines on top of a hypervisor. Those virtual machines might be divided up into containers. The different


The post gVisor Container Isolation with Michael Pratt and Yoshi Tamura appeared first on Software Engineering Daily.

Creating a software company has never been easier. Software engineers are increasingly seeing entrepreneurship as a viable career path. But the path to being an independent software developer is not always clear.  Most engineers spend some of their career working at a software company. Even an engineer who intends to build a company someday can


The post Software Free Radicals with Lior Kanfi appeared first on Software Engineering Daily.

Edge computing allows for faster data access and computation. When your client application makes a request, that request might be routed to the edge. Edge servers are more numerous and more widely distributed than normal data centers, but an edge server might not have all of the data or the complete application logic for the


The post Edge Storage with Steve Klabnik appeared first on Software Engineering Daily.

Today’s episode is a keynote I gave at Full Stack Tech Radar Day in Tel Aviv. The talk is called “You Are Not a Commodity”. This talk is also available as a YouTube video. The slides can be accessed here. The world of commodity engineering is coming to an end. Developers are becoming more productive,


The post You Are Not A Commodity (Keynote at Tikal Full Stack Tech Radar Day) appeared first on Software Engineering Daily.

ListenNotes allows users to find podcasts by categories, popularity, and search queries. ListenNotes is not a podcast client that you download onto your phone, but it allows users to build playlists. These playlists are automatically published to an RSS feed, so that users can curate a playlist on desktop and subscribe to that playlist from


The post ListenNotes: Podcast Search Engine with Wenbin Fang appeared first on Software Engineering Daily.

Open source software allows developers to take code from the Internet and modify it for their own use. Open source has allowed innovation to occur on a massive scale. Today, open source software powers our consumer client applications and our backend cloud server infrastructure.  Linux powers single node operating systems and Kubernetes is the foundation


The post Permissionless Innovation with Joseph Jacks appeared first on Software Engineering Daily.

Google Earth allows users to explore the imagery of the real world. Imagery for Google Earth is taken from satellites, cars equipped with cameras, and other sources. Google Earth renders a data intensive 3-D model of the world on a client application such as a desktop browser or virtual reality system.  WebAssembly is a runtime


The post Google Earth WebAssembly with Jordon Mears appeared first on Software Engineering Daily.

FoundationDB is a multi-model distributed key-value store. It is fully ACID compliant and horizontally scalable. FoundationDB is not usually used directly by an application developer–FoundationDB is a foundational building block for higher level distributed systems such as the metadata store for data warehousing tool Snowflake. Ryan Worl is a software engineer who specializes in FoundationDB.


The post FoundationDB with Ryan Worl appeared first on Software Engineering Daily.

FindCollabs is a platform for finding collaborators and building projects. Three months ago we had our first hackathon, with lots of projects being created and collaborated on. In an earlier episode, we showcased the first place winner ARhythm.  Today’s show features two more interviews with winners from the first FindCollabs hackathon. Kitspace is an open


The post FindCollabs Hackathon Winners: Kitspace and Rivaly appeared first on Software Engineering Daily.

A new database company needs to solve numerous problems in order to succeed. There are already lots of existing database companies, so a new company needs to find a way to strongly differentiate itself. Databases are core infrastructure, so a new database company must earn trust with its customers. A database is a complicated distributed


The post Citus Data: Founding to Acquisition with Umur Cubukcu appeared first on Software Engineering Daily.

Programmers are in high demand, and software engineering is a career path that is fun, creative, and lucrative. There are many people who want to transition into a career in software and are looking for the right path toward writing code. The traditional college computer science curriculum teaches some software engineering skills, but the time


The post Lambda School Education with Andrew Madsen appeared first on Software Engineering Daily.

A grocery store contains fruit, vegetables, meat, bread, and other items that can expire. In order to keep these items in stock, the store must be aware of how much food has been sold and what has gone bad. When a food item is low in stock, the store needs to order more of that


The post Afresh: Grocery Store Software with Volodymyr Kuleshov appeared first on Software Engineering Daily.

Apache Airflow is a system for scheduling and monitoring workflows for data engineering. Airflow can be used to schedule ETL jobs, machine learning work, and script execution. Airflow also gives a developer a high level view into the graph of dependencies for their data pipelines. Chaim Turkel is a backend data architect at Tikal. He


The post Airflow in Practice with Chaim Turkel appeared first on Software Engineering Daily.

Software projects are organized and planned using project management software. Examples of project management software include JIRA, Trello, and Asana. There are hundreds of tools for managing a software project because there are infinite ways that a project could be managed. Google Docs changed project management by allowing documents to be easier to share and


The post Project Management with Kurt Schrader appeared first on Software Engineering Daily.

Niantic is the company behind Pokemon Go, an augmented reality game where users walk around in the real world and catch Pokemon which appear on their screen. The idea for augmented reality has existed for a long time. But the technology to bring augmented reality to the mass market has appeared only recently. Improved mobile


The post Niantic Real World with Paul Franceus appeared first on Software Engineering Daily.

WebAssembly allows for web-based execution of languages other than JavaScript. Programs written in Rust or C++ can be compiled down to WebAssembly and shipped over the browser for on-the-fly execution in a safe, memory controlled environment. WebAssembly has been in development for more than two years, and is still an immature ecosystem because building the


The post WebAssembly Compilation with Till Schneidereit appeared first on Software Engineering Daily.

Sheng Liang was the lead developer on the original Java Virtual Machine. Today he works as the CEO of Rancher Labs, a company building a platform on top of Kubernetes. Sheng joins the show to discuss his experiences in the technology industry. The container orchestration wars had many victims. The competing standards for how an


The post Infrastructure Wars with Sheng Liang appeared first on Software Engineering Daily.

Kubernetes has made distributed systems easier to deploy and manage. As Kubernetes has become reliable, engineers have started to look for higher level abstractions we can define on top of Kubernetes. An operator is a method of packaging, deploying, and managing a Kubernetes application. Operators are useful for spinning up distributed systems such as Kafka,


The post Kubernetes Operators with Rob Szumski appeared first on Software Engineering Daily.

Cloud computing was popularized in 2006 with the launch of Amazon Web Services. AWS allowed developers to use remote server infrastructure with a simple set of APIs. But even with AWS, it was still not simple to deploy and manage a web application. In 2007, Heroku launched a platform built on top of AWS. Heroku


The post Render: High Level Cloud with Anurag Goel appeared first on Software Engineering Daily.

Software engineering is an art and a science. To manage engineers is to manage artists and scientists. Software companies build practical tools like payment systems, messaging products, and search engines. Software tools are the underpinnings of our modern lives. You might expect this core infrastructure which modern humans rely on to have been constructed with


The post Elegant Puzzle with Will Larson appeared first on Software Engineering Daily.

Machine learning allows software to improve as that software consumes more data. Machine learning is a tool that every software engineer wants to be able to use. Because machine learning is so broadly applicable, software companies want to make the tools more accessible to the developers across the organization. There are many steps that an


The post Stripe Machine Learning Infrastructure with Rob Story and Kelley Rivoire appeared first on Software Engineering Daily.

The software market changes every year. As individuals and enterprises become more willing to buy software, there are new markets for entrepreneurs to sell software to. Good software has high margins and high retention, so even a niche software business can prove profitable. As software spreads across the world, developing countries are showing a willingness


The post Software Businesses with Hiten Shah appeared first on Software Engineering Daily.

Google Cloud was started with a vision of providing Google infrastructure to the masses. In 2008, it was not obvious that Google should become a cloud provider. Amazon Web Services was finding success among startups who needed on-demand infrastructure, but the traditional enterprise market was not yet ready to buy cloud resources. Googlers liked the


The post Kubernetes Vision with Joe Beda appeared first on Software Engineering Daily.

Containers are made to fail gracefully. When your container shuts down due to a hardware or software failure, your distributed application should be able to tolerate that failure. One simple way to be able to tolerate such a failure is to make all of your application logic “stateless.” If your application does not maintain state,


The post Kubernetes Storage with Saad Ali appeared first on Software Engineering Daily.

Amazon Web Services is the leading cloud provider by a large margin. Amazon established its lead by being first to market in 2006, with Google and Microsoft taking several years to catch up to the huge business opportunity of the cloud. Since 2008, Google Cloud has been working on cloud products for developers. It started


The post Kubernetes Market with Adam Glick appeared first on Software Engineering Daily.

Containers offer a lightweight abstraction for running a server. Cloud providers are able to manage billions of containers from different users, allowing for economies of scale so that each user can pay less. Today, there is a variety of ways that users can deploy containers on a cloud provider. These containers can run in managed


The post Service Mesh Interface with Lachlan Evenson appeared first on Software Engineering Daily.

Each cloud provider offers a different set of services which are not always compatible with each other. What are the challenges of building an application that interoperates with multiple different clouds? The first issue is API compatibility. Most cloud providers have a managed SQL offering, a bucket storage system, and server abstractions like virtual machines


The post Multicloud Future with Bassam Tabbara appeared first on Software Engineering Daily.

Kubernetes has evolved from a nascent project within Google to a thriving ecosystem of cloud providers, open source projects, and engineers. Tim Hockin is a principal software engineer who has been with Google for 15 years. Tim joins the show to talk about the early days of the Kubernetes projects, and the engineering efforts that


The post Kubernetes Development with Tim Hockin appeared first on Software Engineering Daily.

Google’s cloud business was long regarded as a place where startups could build a business, but not established enterprises. For serious workloads, enterprises chose Amazon almost unanimously. This phenomenon of Amazon as the default was described by a phrase that harkened back to the days of IBM’s dominance: “nobody ever got fired for choosing AWS.”


The post Google Anthos with Aparna Sinha appeared first on Software Engineering Daily.

A service mesh is an abstraction that provides traffic routing, policy management, and telemetry for a distributed application. A service mesh consists of a data plane and a control plane. In the data plane, a proxy runs alongside each service, with every request from a service being routed through the proxy. In the control plane,


The post Service Mesh Wars with William Morgan appeared first on Software Engineering Daily.

Cloud computing, open source, and mobile computing are trends that affect every organization. When a large organization adapts to these trends, it is commonly referred to as a “digital transformation”. Digital transformation causes many companies to reframe their business as a software company. A candy manufacturer now must think of itself as a software company


The post Digital Transformation: Capital One with Hillary McTigue appeared first on Software Engineering Daily.

Netflix started with a DVD-by-mail product. The software infrastructure and operations practices needed for the DVD business were very different from those needed by a streaming video company. Since the early days of Netflix, CEO Reed Hastings knew that the company would evolve to becoming a streaming video platform. But he did not know when


The post Netflix Early Days with Greg Burrell appeared first on Software Engineering Daily.