NICOLÁS BOTTARINI
Digital House Certified Tech Developer

Digital House Certified Tech Developer

Year:2020 - 2022
Industry:
EdTech
Deliverables:
webbackend
Technologies:
KotlinPostgreSQLJavalinjOOQTypeScriptReactAWSECSAuroraRedisTerraformRabbitMQCloudWatch
Technical focus:
TDDContinuous IntegrationContinuous DeliveryPair ProgrammingDDDClean ArchitectureE2E Tests

Introduction

Certified Tech Developer was a large-scale educational project launched by Digital House together with Mercado Libre and Globant, with the goal of training new technical profiles in Latin America through an intensive program in programming and digital skills.

We worked with Digital House on one of the most important challenges of the program: scaling its onboarding process to reach thousands of applicants in different countries. We built a platform that automated the whole candidate onboarding flow, from enrollment to scholarship assignment and access to the virtual campus.

The first launch had a publicly committed date and a massive media campaign, so we knew it was going to receive a very high volume of traffic from day one. It was critical to guarantee the stability of the platform. That's why we ran stress tests and optimized the key services to make sure it performed well under load.

My role was architecture, technical leadership and hands-on development. I designed and implemented the core components of the solution and worked with the team to make sure every technical decision was aligned with the business goals, the expected quality and the project deadlines.

The impact was concrete: campaigns were launched in 7 countries, more than 100,000 enrollments were processed and 5,000 scholarships were granted, contributing directly to the regional positioning of the program and to the training of new digital talent.

Screens

The launch had a lot of media coverage. This is the presentation video of the program.

Media coverage
The launch in Clarín, Infobae and Globant

Main features

The platform automated the complete candidate onboarding process, from the initial enrollment to the final scholarship assignment. It was designed to be modular and configurable, so it could adapt to the changing needs of each campaign and country.

It allowed creating dynamic enrollment forms, running automatic evaluations through quizzes and technical tests, and managing video presentations, scheduled interviews and gamified evaluations with artificial intelligence. Each step contributed to a scoring system that fed a rules engine for scholarship assignment, taking into account quotas and specific characteristics of the applicants.

It also included online payment processing and the simultaneous management of multiple campaigns and cohorts with custom workflows.

The platform was integrated with several internal systems of Digital House: the CRM for candidate follow-up, the academic system for enrollment, Power BI for real-time data analysis, and the Treasury system to validate payments and grants. The whole ecosystem worked together to offer an agile, scalable and traceable experience.

Technical approach

The Certified Tech Developer platform was built as a robust and scalable solution to automate the onboarding process of thousands of candidates in simultaneous campaigns across the region.

I chose Kotlin as the main backend language for its expressiveness, productivity, and balance between object-oriented and functional paradigms. It allows writing clear and declarative code, without Java's verbosity.

Libraries over frameworks

I avoid using frameworks, as they introduce a lot of indirection, unnecessary complexity, and take control over the application (it's the framework that ends up orchestrating it). Instead, I prefer lightweight libraries and a simple, explicit, and sustainable design.

For the web layer of the API, I used Javalin, a minimalist library that wraps Jetty, ideal for keeping tight control over request flow without hidden behaviors. With no framework in the middle we had direct access to Jetty, which allowed us to adjust its configuration to handle the load of the launch.

Stress tests

The launch had a fixed date, announced in the media, and the backing of Mercado Libre and Globant. Before going live we needed to know how much load the platform could handle and where it would break.

I built the scenarios with Artillery and we ran them in parallel from several EC2 instances against the staging environment (a single machine was not enough to generate the load). We iterated: run the test, look at the metrics in CloudWatch and Performance Insights, adjust and run again, increasing the load on each round.

First we tested the loading of the enrollment form, which was the first thing that received the traffic peak:

Stress test results
Enrollment form loading: evolution between iterations

The changes with the biggest impact were:

  • Database indexes and JVM tuning.
  • Tuning of the Jetty thread pool (from 250 to 1,200 threads) and a management port with its own thread pool, so the ECS health checks didn't compete with real traffic.
  • Horizontal scaling: with 4 API instances we processed almost 130,000 requests per minute with no errors and with the API CPU below 40%.

Then we tested the complete enrollment flow. There the bottleneck moved to the database: with a single API instance we processed around 6,250 enrollments per minute and the database CPU reached 97%. The solution was to move to an instance with more CPU.

Stress test screenshots
Monitoring during the tests

Persistence

For persistence, I ruled out the use of ORMs: they introduce over-engineering, obscure SQL, and promote an inadequate model for rich domains. Instead, we used jOOQ, a typed SQL DSL that allows full control over queries, integrating it with the Repository pattern to encapsulate data access in a clear and maintainable way.

ORMs usually map tables to data structures, not to rich domain objects. I prefer to explicitly separate layers and model the domain with intention.

We used PostgreSQL as the database and Flyway for schema versioning. We used read-only replicas to distribute the load. We implemented CQRS: write commands operated on the main database, while reads were sent to replicas configured specifically for that load. To take even more load off the database we added Redis as a cache for the most frequently used entities.

Explicit dependency injection

I don't use dependency injection containers. With a clear architecture, manual injection is simple and avoids unnecessary indirection. This improves system readability and makes execution flows easier to trace.

Auxiliary tools

I developed a CLI tool in JavaScript: a development console that automated frequent tasks and helped us improve productivity. This CLI also allowed us to run remote maintenance commands in the different environments (campaign administration, data imports, etc.). I also created PopulateDB, a utility to initialize the development database with different scenarios and contexts, which made it easier to develop and validate features in different use cases.

CI/CD and deployment

We used AWS as the cloud platform, with ECS for container orchestration, ECR for Docker images, CloudWatch with custom dashboards for monitoring, RabbitMQ, Redis and Aurora as the database. We automated the full provisioning of the staging and production environments with Terraform, including service configuration, networks (VPC), firewall rules and dependencies.

The CI/CD pipeline was implemented in GitHub Actions and included:

  • Automatic execution of unit, integration and end-to-end tests
  • Build of versioned Docker images and upload to ECR
  • Automated deployment to ECS
  • Execution of Flyway migrations when needed
  • Full provisioning of staging on every infrastructure change
  • Manual approval step for production, with Terraform diff preview before applying changes
CI/CD Pipeline
CI/CD Pipeline
Infrastructure
AWS infrastructure

Monitoring and operation

We created multiple dashboards in CloudWatch for real-time monitoring of the system, including infrastructure metrics, service health, event queues and resource usage.

Monorepo

We used a monorepo to group all system components: backend, web frontend, backoffice and internal tools. This greatly simplified local setup (done with a single command), favored synchronized versioning across components, and made full-stack development easier.

Architecture

The backend was a modular monolith. Each module corresponded to a Bounded Context (admission, identity and access, content, reporting and tracking) and had its own API and its own core. They were deployed together, which saved us from the operational complexity of microservices without losing the separation between contexts.

Inside each module I applied Domain-Driven Design and organized the code following the principles of Clean Architecture.

Architecture
API Architecture
  • The candidate application and the backoffice communicated with the HTTP API, which acted as the delivery mechanism.
  • The controllers were lightweight: they interpreted the request and translated it into a command or query.
  • A bus routed each command or query to the corresponding handler (use case).
  • The worker (which processed events), the ECS scheduled tasks and the maintenance CLI entered through that same bus. All of them executed the same use cases; there was no duplicated logic depending on the entry point.
  • The domain held the core of the system: entities, value objects, domain services and events.

The integrations with the Digital House systems, the repositories and the event publishing lived in the infrastructure layer. Since dependencies were inverted (infrastructure depended on the domain, not the other way around), the domain didn't know which external system it was talking to.

Domain events were published to RabbitMQ and the worker processed them asynchronously. This allowed us to take out of the request everything that didn't need to be immediate, like notifications.

In the frontend I used the same architecture as in other projects: React as a UI library and the view logic in Presenters, pure TypeScript classes with no dependency on React.

Testing strategy

We applied TDD across the whole project. The backend tests were split into two groups:

  • Fast tests: use cases and domain, using test doubles for the infrastructure (for example in-memory repositories).
  • Slow tests: integration tests on the real infrastructure, for example the jOOQ repositories against a real PostgreSQL.

The pipeline ran both groups on every commit before deploying.

Work methodology

I worked with cross-functional teams that included all the roles needed to build a product: Product Owner, graphic design, and full-stack developers. Developers were generalist-specialists: while some had deeper expertise in web, backend, or infrastructure, everyone could contribute across the entire codebase.

We worked daily using pair and mob programming. This ensured knowledge flowed continuously and helped everyone quickly understand all aspects of the system.

We practiced TDD and continuous integration. We didn't use feature branches in Git: we made micro-commits as part of the TDD cycle, and those were integrated directly into GitHub, where a pipeline ensured everything was in order.

If something failed on our only branch (main), the whole team prioritized fixing it. Main was always deployable. Every commit that passed the pipeline was automatically deployed to the staging environment, so stakeholders could review the latest changes and give daily feedback.