Business need
Democratizing access to data
Data supports Itaú’s decision-making at all levels of the organization. It’s core to the bank’s culture. However, access to key data about the bank administers or manages was largely limited to employees who worked in Investment Services.
To improve decision-making, Itaú needed to make this information available to other business areas, analysts and operational staff. The goal was to make it easy for people to find, cross-reference and use relevant information, without long wait times or a reliance on specialized data teams, and with strong data security and governance in place. To democratize data in this way would reduce the manual effort of searching for information in different systems, consolidating it and reconciling inconsistencies before use.
“We faced the challenge of accelerating our data democratization agenda within the business,” says Ellen de Vasconcelos, Technology Manager for Itaú’s Investment Services. “The intention was to give our internal users access to near-real time information, consolidated in a data lake.”
To allow the business to respond quickly to market demands and the needs of key internal users — and ultimately support simpler, faster and more consistent customer journeys — a more distributed and accessible data governance model was essential.
Architecture modernization
Itaú built this initiative on a data architecture that used a data-mesh approach. A data mesh is built on four pillars: domain data ownership, data as a product, a self-service data platform and federated governance.
This approach shifts responsibility for data closer to the business domains, treating data as a product and making it easier for teams to access information that is relevant to them. It balances wider access with strong governance and clear standards for data security, privacy and regulatory compliance, including alignment with Brazil’s General Data Protection Law (LGPD).
This foundation highlighted the need for a dedicated technical architecture designed around events and modular components.
Challenges
Limited data access and slow updates
Before modernization, the bank’s Investment Services business faced challenges in making data available and keeping it up to date. Delays slowed the flow of information to teams that depended on it and reduced operational efficiency.
Fragmented data sources
Data was spread across different sources and formats, each with its own controls. This made data integration and standardization difficult, limiting the bank’s potential to generate broad, comparative analyses.
According to Kelly Nascimento, IT Director at NTT DATA, a unified approach to data intake and distribution was critical to improving the consistency and reliability of information. Alexandre Gusson, Director of Business IT and Strategic Alliances at NTT DATA, adds that a new model was also required to standardize metrics and build performance indicators (KPIs) to support more accurate comparisons and consistent insights across the organization.
We chose NTT DATA for their ability to bring together technical expertise in data architecture with experience in large projects in the financial sector. The partnership brought fluidity to complex stages and focus on business results.”
Solution
Implementing a new architecture
We followed a phased and collaborative approach that combined NTT DATA’s technical expertise with the bank’s strategic vision for their platform and data domains.
The first step was to identify and prioritize the data domains that could deliver the greatest business impact. From there, the teams designed an event-driven architecture that can process and orchestrate data in near-real time. This improved control over data updates, consistency and traceability.
We implemented a pilot focused on a single domain so the teams could design the full data flow before scaling the project. The pilot involved everything from data catalogs, application programming interfaces (APIs) and dashboards to establishing clear adoption metrics.
“This approach was made possible with Apache Kafka, which acts as a communication backbone between systems and efficiently distributes data across the different layers of the data mesh,” explains Michel Gobbato, Head of Financial Services at NTT DATA. “As a result, Itaú now has a highly responsive and scalable data flow.”
Using an open table format
The adoption of open table formats, especially the Apache Iceberg format, played a key role in helping Itaú govern large volumes of data in a more flexible and reliable way. Iceberg offers capabilities such as version control, schema evolution and query optimization.
“We were one of the first business areas to adopt open table formats and the Iceberg table format,” says Ellen de Vasconcelos. “This allowed us to build an event-flow-oriented architecture in Kafka and a microservices-based data ingestion pipeline.”
These capabilities made it possible to update data incrementally, speed up complex queries and improve reliability in data processing. This pipeline has become the standard approach for bringing data into the platform and making it available for analysis in the bank’s Investment Services business.
Microservices orchestration with AWS Step Functions
To improve efficiency and modularity across the data ingestion flow, Itaú adopted AWS Step Functions to coordinate the microservices involved in reading, transforming and enriching data. This visual workflow service makes it easier to design, manage and maintain distributed applications, while giving teams greater visibility and control.
With this service, the bank significantly reduced response times and strengthened the resilience of its architecture. The environment is now better prepared to scale and grow as business needs change.
The creation of Orange Flow, a replicable framework
By combining Kafka, Iceberg and AWS Step Functions, Itaú developed Orange Flow — a data flow framework designed to be used by different teams and data communities within the organization.
Orange Flow makes it possible to standardize and automate data collection and distribution processes, paving the way to apply the same approach in other areas of the bank.
The solution is based on managing system-wide flows, with real-time monitoring and a visual workflow that makes it easy to build, manage and scale distributed applications.
“The decision to create our own product was not made lightly,” says Fabio Rocha, creator of the product and Data Engineering Chapter at Investment Services. “We evaluated several market solutions but ultimately concluded that developing a technology aligned with our chosen cloud model and tailored to the technical and business needs of our teams made more sense in the medium and long term.”
The solution includes continuous monitoring, with KPIs focused on usage and access, as well as regular review cycles to refine both the architecture and governance. Orange Flow was designed to scale reliably while staying closely aligned with business requirements.
Outcomes
Faster access to data
With the new architecture in place, Itaú reduced the time required to make data available by about 70%. Transactional data reaches the data mesh in less than 10 minutes, supporting faster responses to business and customer needs.
Large-scale processing efficiently
The Investment Services business can process more than 650,000 events every day and orchestrate about 2 million flows using AWS Step Functions. This reduces rework and, most importantly, instills greater confidence in the reliability of the information being used.
Democratization of information, with governance
“More than 3,100 tables have been democratized, representing about 40% of the most relevant data for the business,” says Ellen de Vasconcelos. All this information is made available with the appropriate layers of governance, privacy and security.
Direct impact on customer experience
Improved architecture, better access to data and the resulting operational efficiency translate directly into a better customer experience. The greater accuracy and timeliness of the information allow the bank to offer more personalized, flexible solutions aligned with investor needs. With automated processes and accessible data in near-real time, Itaú strengthened its leadership position by offering services that are smarter, more efficient and customer-focused.