This has been a year of recovery and resurgent growth in the tech industry. And with deadlines approaching and targets to meet, engineering teams across the country are busy finalizing end-of-year tasks and setting themselves up for the year to come.
In our work lives it’s easy — even natural — to always look to the future and focus on the challenges to come, rather than reflect on past accomplishments.
So before we break for the holidays and come roaring back in 2022, we asked six tech leaders around the country to reflect on their team’s achievements thus far.
Company background: Bolt integrates with e-commerce platforms to provide one-click purchase functionality. Meanwhile, the company’s SSO Commerce platform replaces username-and-password login combinations with one-time passwords, giving users instant access to all merchants within the Bolt network.
In the news: Riding a pandemic-induced acceleration of existing online shopping trends, Bolt recently unveiled a $393 million investment at a $6 billion valuation.
Aside from the stresses of the pandemic and working from home, what was the biggest work challenge or project facing your team at the beginning of 2021?
Over the past few years, Bolt has developed a frictionless login and registration flow that has helped us grow our network to more than 10 million accounts. In 2021, one of the key priorities for our team was to let merchants leverage this infrastructure to enhance their own storefront account experience. This would let Bolt account users create and access their storefront accounts with one click while still maintaining the same level of security and privacy. This was a monumental challenge from both a technical and UX perspective. A few pieces that were particularly challenging involved creating secure and robust logic for linking the two accounts; populating the login session with the shopper’s in-progress order; allowing account creation outside of checkout for the first time; and building our own OAuth resource and authentication server.
“Thousands of shoppers have used SSO Commerce to create merchant storefront accounts, giving them access to benefits like order tracking, wish lists and loyalty programs.”
Did that challenge or project evolve throughout the year?
Bolt SSO Commerce federates disparate account logins with Bolt network accounts, creating a single digital identity for shoppers. As we began to build SSO Commerce, we discovered the need to let shoppers create an account with just an email. Bolt’s account system was originally built under the assumption that every account would have a phone number, so we had to rethink a lot of that architecture while still maintaining the highest level of security. An unexpected benefit of this was that we opened the door to a number of other UX improvements that we can build for our shoppers and helped further reduce any login friction by limiting the number of required fields.
What was the ultimate result of that work?
Today, SSO Commerce is live with dozens of merchants including Lucky Brand, Badgley Mischka and the Hardwood Lumber Company. Thousands of shoppers have used SSO Commerce to create merchant storefront accounts, giving them access to benefits like order tracking, wish lists and loyalty programs. Plus, they get the speed and security of Bolt’s one-click checkout experience across all sites in the Bolt network. There’s no other network account system in the world that seamlessly unites with storefront accounts to benefit both merchants and shoppers, and we’re excited to keep pushing the boundaries of what is possible.
Company background: It’s one thing for IoT devices, applications, computers, networks and containers to produce data — it’s quite another to actually make sense of it. To that end, InfluxData pioneered open-source time series database technology with InfluxDB, which helps organizations build applications for real-time analytics with a minimal amount of coding.
A busy year: In 2021, the company’s multi-tenant SaaS development platform for time series data – dubbed InfluxDB Cloud — saw tremendous traffic growth. Every few months, Dix said, the number of queries running on the platform doubled. InfluxDB Cloud now handles roughly 1.3 billion queries per month — and counting — and this traffic growth presented some major challenges for the team.
What was the biggest work challenge or project facing your team at the beginning of 2021?
At the beginning of the year, multiple customer queries were executed within the same container. This was fine until we had so many customers running queries that their queries began to compete for resources within a container. This “noisy neighbor” problem would cause queries to time out or run out of memory unpredictably. Worse, if a customer’s query caused a crash, it would take out other customers’ queries along with it. The solution was to implement a queue, and then to ensure that each query was run in an isolated environment inside a container. After resolving the “noisy neighbor” issues, we were then able to go one layer deeper and identify performance bottlenecks throughout our Kubernetes infrastructure. For example, we were able to carefully optimize the code between the query execution engine and the storage tier. This ensured that the query engine was able to interleave segments of data from the storage engine as the data arrived, rather than reading one segment at a time. Additionally, by working closely with our SRE team, our developers were able to reduce network latency in the platform.
Throughout the year, as our user base of developers grew, they dreamed up new and creative ways to use InfluxDB Cloud. In some cases, these workloads had surprising results, which is to be expected for a distributed system like ours. Working with our support team, we were able to understand the users’ workloads and improve reliability by adding rate limits and system protections to stop “runaway” workloads before they caused problems for other users.
“Users are now able to query their data faster and with fewer errors.”
Did that challenge or project evolve throughout the year? And if so, how?
User demands for availability and performance are always increasing. At the start of the year, reliability was defined as queries not returning errors and that was considered good enough. Now we define reliability as queries not only returning quickly and without errors but also having performance be predictable and consistent for users.
What was the ultimate result of that work?
By the end of 2021, InfluxDB Cloud has seen a drastic reduction in the number of incidents and incident severity while simultaneously improving the performance and efficiency of the platform. In short, users are now able to query their data faster and with fewer errors — meaning development teams can work even faster, with peace of mind knowing that, as they scale their application building, our technology can scale with them.
Company background: Cash App was a peer-to-peer payment pioneer, launched out of fintech company Square in 2013. The team has since branched out to offer banking and investment services.
On the horizon: Cash App’s engineers are both driving and adapting to rapid changes in the consumer finance industry. In 2022 the company plans to build in-app commerce options like “buy now, pay later” as well as expand its existing banking and cryptocurrency features. The company is also making heavy investments in scaling its infrastructure to support big growth targets.
What was the biggest work challenge or project facing your team at the beginning of 2021?
Every year, we put together a list of challenges that our engineering team faces to share with the broader team. This year, those included scaling our processes to support the growth of a diverse organization, ensuring product velocity stays high and decentralizing monolithic services.
However, if I were to pick just one challenge, I’d orient around ensuring that we’re continuing to best support our customers as they use our products. As we’ve moved from a peer-to-peer app to one that customers trust with more of their financial lives, we need to ensure we’re building the right level of credibility with them. Trust is hard to earn and easy to lose, so we’re leaning in heavily to this space.
The scope of this work covers many things. For instance, we want to give customers great and timely support through channels that work for them — whether that’s email, in-app or over the phone. We also want to maximize great experiences and minimize the pain points that our customers may face. Lastly, we want to focus on giving customers options when they run into trouble.
“We’ve improved on our KPIs, overhauled internal tooling, expanded product offerings to include P2P bitcoin, paper money and check deposits and released in-app chat-based support for all customers.”
How did that challenge or project evolve throughout the year?
There’s always more work to be done. One of the key challenges we’ve been facing has been the decentralization of our monolithic services. As part of this initiative, we’ve undertaken several pieces of work to decentralize our identity verification, limits, aliases like phone and email, financial instrument code and data from our main monolith in order to better support changes in the product.
What was the ultimate result of that work?
We’ve achieved so much. We’ve improved on our KPIs, overhauled internal tooling, expanded product offerings to include P2P bitcoin, paper money and check deposits and released in-app chat-based support for all customers. All of these achievements ultimately allowed us to release Cash App for anyone over the age of 13 this November.
Company background: Textio helps companies build and evolve inclusive employer brands, with technology that guides language choices as users write out job posts, email, career sites and employee blogs. The idea is to identify and eliminate language choices that may tend to exclude already marginalized people and identify phrases that may, for example, play on harmful age- or gender-based biases.
A new product: In addition to its core product, Textio is currently working on Umeo, a tool that crowdsources meeting agendas to check whether participants are aligned, with the goal of fostering more focused and productive working sessions. Umeo is in a closed alpha phase right now, but moving quickly toward a beta in early 2022.
What was the biggest work challenge or project facing your team at the beginning of 2021?
This year, we set out to design and build a totally new communications product called Umeo. Building on the trend of writing apps more natively on AWS, we used this opportunity to invest in a number of technologies that were new to our stack, including AppSync, Apollo, GraphQL, CDK and Reakit. This was in addition to many technologies we use every day, like AWS Lambda, TypeScript, React and CloudFormation. We hit challenges along the way, like learning the hard way that AppSync caching can introduce subtle and unexpected bugs.
“Integrating Textio into more places that people write is particularly challenging, as our browser plugin has to adapt to all sorts of DOM layouts, page designs and workflows in a coherent way.”
How did that challenge or project evolve throughout the year?
One takeaway was a reminder that working on multiple products at once can be challenging, especially as a newly distributed team hiring in eight U.S. states. Some of the processes we use to keep the Textio flagship platform working well are too heavy for the earlier phase Umeo is in. So we went back to first principles and created new ways of working together with shorter time horizons, quicker iterations and tight, continuous interactions between engineering, design and product.
What is your team’s biggest or most exciting challenge for 2022?
Integrating Textio into more places that people write is particularly challenging, as our browser plugin has to adapt to all sorts of DOM layouts, page designs and workflows in a coherent way. We founded the company to build products that create understanding between people, and between these new integrations, ML/NLP models to support new kinds of writing and language support and a host of new platform features that span UX to infrastructure, we’re poised to design and build some of the most impactful software of our lives.
Company background: North American Bancard offers credit card processing, e-commerce, digital gateway, mobile payments and cash advance services for businesses and merchants of all sizes.
A big year: NAB’s tech team was charged with executing the final stages of the company’s cloud migration and transformation to a microservice-based architecture in 2021. Latif’s team used the project as a learning opportunity, empowering engineers to research and present ideas to the wider team.
What was the biggest work challenge or project facing your team at the beginning of 2021?
In recent years, there have been a lot of changes in the electronic payments industry. We needed to build a lot of products and services to support our growing number of internal and external customers. We also had to focus on making our systems more scalable, efficient, secure and maintainable. We believed that the most efficient way to support these goals was to complete our cloud migrations and microservice architecture transformation, which was our main focus for 2021.
“We had ‘show and tell’ sessions where engineers presented their solutions and tools and how others can be benefited.”
How did that challenge or project evolve throughout the year?
When we started our journey to the cloud, our approach was to “shift and lift” without making modifications to existing applications. As our engineers trained and learned more about available cloud-native tools and services, we started adopting these new technologies to make our systems more robust. With our continuous support of learning and development, we partnered with a cloud training platform and provided all engineers with opportunities to learn and try different cloud tools. We also had “show and tell” sessions where engineers presented their solutions and tools and how others can be benefited.
What was the ultimate result of that work?
The ultimate result was transforming our older monolithic architecture into microservices. Right now, we have teams responsible for a microservice for all steps of our systems development life cycle. Engineers who worked on designing, developing and deploying continue to maintain and provide everyday support to their applications. This allows engineers to work on different components and with different technologies. This promotes a culture where talented engineers are working, learning and growing together. Through this transformation, we have also understood the importance of the system architects’ role and now have a system architects team that works closely with developers on systems design and architecture.
Company background: Wonder prepares meals in centralized kitchens, before chefs in customized mobile kitchens finish and serve them at a customer’s home. The idea is to combine the convenience of eating at home with the experience of fine dining in a high-end restaurant, avoiding the pitfalls of soggy vegetables or limp french fries because everything’s been whipped together right outside the customer’s door.
Institutional expertise: Wonder is led by Scott Hilton, who previously led revenue teams at e-commerce company Jet.com. After that business was acquired by Walmart, Hilton stuck around to become chief revenue officer for the retail giant’s e-commerce unit. On the engineering side, Schaeffer has followed a similar path, moving to Walmart through the Jet.com acquisition and then starting with Wonder last year.
What was the biggest work challenge or project facing your team at the beginning of 2021?
Despite many of the challenges that most organizations faced this past year, Wonder had a year full of growth — from organizational growth to continued product development and iteration. Wonder operates in a space that is brand new in many aspects of our business, and we are often faced with challenges to which we do not fully know the solution. Product development at Wonder involves research, development, measurement and iteration.
Personally, my biggest tech challenge has been the implementation and rollout of a fleet telematics system that is responsible for understanding what is happening inside of our many mobile kitchens. This includes the obvious data points such as location and environmental signals but extends to many other factors such as our appliances. I like to think of it as the mobile kitchen operating system. The cross-functional nature of this project brought together software engineering, engineering, operations and product.
“As an early stage startup, we are pulling out all stops and growing our teams as we plan for scale.”
How did that challenge or project evolve throughout the year?
As with all things in software and product development, our assumptions are tested and often invalidated. This feedback loop is important to our product development cycle as the idea of continuous improvement is core to our ability to react to customer and operations feedback and meet our goals. In this particular case, we have physical limitations such as space, power constraints and weight constraints. Putting devices in a harsh environment where cooking and driving are happening makes you quickly realize how important it is to test hardware for reliability and durability. To adapt, we’ve built a process to vertically integrate new ideas, software, hardware and processes to a subset of the operation, allowing us to move fast, test ideas and iterate.
What is your team’s biggest or most exciting challenge for 2022?
Thoughtful scale! As an early-stage startup, we are pulling out all stops and growing our teams as we plan for scale. Operating a fleet of mobile kitchens, food production facilities and a distribution network all take technical innovation and operational creativity. As we grow our business, there are many technical problems to solve such as expanding on the functionality of the fleet telematics system and managing a distribution network responsible for moving around high-turnover inventory all while serving high-quality, hot, fresh food to our customers.
