Developer Blog Series Vol.9 - Data Visualization and Analysis -
Vol.09 Data Platform and DX Promotion
This is the ninth article on data analysis and visualization.
Why do we need data integration platform and a data utilization platform to promote DX?
I think there are few people who can accurately explain why a data infrastructure is necessary to achieve DX. To explain this, I will quote the Susanoo Framework from the IPA's "DX Practical Guide: IT System Construction Edition."
The Susanoo Framework represents the set of technical elements that realize IT systems in a company, and summarizes the technical elements that have been proven effective through research into previous cases.
"Susanoo Framework" - a set of technical elements that realize the ideal IT system
In the picture, the technological elements are assigned letters from A to J.
I think the elements that are easiest to imagine are (A), (B), (H), and (I).
If we divide the applications within a company into the following two-axis matrix, the mapping will be as shown in the table below.
- Areas that will become the source of competitiveness/areas that will promote efficiency and standardization
- Areas that are customer contact points/areas related to internal business processes
| Areas related to internal business processes | Customer contact areas | |
|---|---|---|
| Areas that are the source of competitiveness | Business and mission-critical system, core system (B) (Each company's mission-critical system, core system) |
Each company's own services/apps (A) (Membership sites, e-commerce sites, etc.) |
| Areas for promoting efficiency and standardization | Industry Common Platform (I) (e.g. Concur) |
Collaboration with other industries/External services (H) (External payment services, etc.) |
It is also easy to imagine a data utilization platform (D) and a data analysis platform (E).
This represents an environment that enables data to be collected from inside and outside the company and utilized, and corresponds to our Data-Driven Platform (hereinafter referred to as DDP).
It is easy to see why having external information such as weather information and map information when analyzing data would broaden the scope of analysis. (External Data Source (J))
Let's think about the remaining (C) and (F).
What does intermediary layer (C)/API (F) refer to?
The following are the IT system requirements for achieving the "speed agility" that underpins the Susanoo Framework:
A system suitable for implementing DX is one that can independently, agilely, and safely update applications/programs in response to individual changes in business models and services.
(Omitted)
When connecting groups of applications that are loosely coupled and independent on a functional basis, it is necessary to devise ways to connect them "loosely" without tightly coupling them, so as not to place a burden on the newly constructed DX platform/applications and mission-critical system, core system.
What I mean is,
- The mechanisms mentioned above (A), (B), (H), and (I) need to be independent.
- However, since information needs to be passed, let's link them in a loosely coupled manner.
That's what I'm saying.
This role is played by the intermediate layer (C)/ API (F).
This loosely coupled integration creates an environment where each element other than (C) and (F) can be developed and operated individually without slowing down.
To put this in our terms, the intermediary layer (C) is data integration platform, and the API (F) is iPaaS.
- The explanations are given with a priority on ease of understanding, so they may not be correct in all situations.
- Note that the term API does not refer only to REST APIs.
Based on the explanation so far, the diagram I presented at the beginning looks like this:
Why "foundation"?
For example, dashboard development for a company that does not have a data analysis infrastructure will end up being a one-off project.
The same is true for interface development projects that lack data integration platform.
For example, if someone decides they want to analyze something, would that be considered DX if they had to write a request form, get approval, create the environment, and take six months to complete the process?
If you have to build the foundation every time you want to do something, things will never move forward.
When you want to write a blog post, if you have to start from scratch with HTML for each post, you probably won't feel like writing.
Dashboards and interfaces are content.
It is important to separate the content from the foundation on which that content resides, so that whenever you want to post content you can do so immediately.
Separating the platform and content will mean a slightly different approach for those with a long history of SI development.
First, the foundation is laid, and then (or in parallel) multiple content developments begin to move forward asynchronously.
This is what makes it different from normal SI development and is a distinctive feature, so it may be a good idea to remember it as prerequisite knowledge.
DX maturity and the mindset required
The progress of DX and the amount of effort it is putting into it vary from company to company.
It is important to consider the current situation, but it is also important to draw up a roadmap for how to get DX on track.
A useful framework for thinking about such things has also been published by the IPA, so I will quote it here.
"DX maturity level."
The diagram below shows the approach and points that are emphasized based on my own experience.
The early stages of DX are primarily bottom-up efforts.
It is also important to produce small but early success stories so that further efforts can continue.
What is important is agility.
On the other hand, as the level of maturity increases, a top-down, coordinated approach becomes important.
This is because grassroots activities alone inevitably have a limited impact on business performance.
As the initiative scales across the entire company, ROI and other factors will likely become more important than creating small success stories.
In order to make this an effort with control, management/governance (rules and frameworks) becomes increasingly important.
Now, is there anyone who is confused by this explanation?
When you read it over and over again, you will notice that a paradox arises in the process of increasing DX maturity.
- In the early stages of DX, spending too much time on management/governance (i.e., creating rules) will result in a loss of agility.
- Ignoring management/governance at the start will hinder the maturity process.
In other words, this leads to the question of when exactly the rules should be created.
The solution will be discussed in the next chapter, so first remember that while you need to create rules, you must not lose agility.
Stack of foundational components
The foundation components explained so far can be organized into a stack as shown below.
The structure is that there is a foundation, rules are placed on top of that, and content is layered on top of that.
The specific details of what this means are noted on the right.
If we overlay the lineup of services we offer on this, it looks like the diagram below.
This leads to a solution to the paradox mentioned at the end of the previous chapter: when exactly should rules be created?
To ensure that rule-making (or the lack of rules) does not become a hindrance in the process of increasing DX maturity, we will create the minimum number of rules using our know-how with the minimum amount of effort.
The service that makes this possible is the "concept design" that we offer.
- To be precise, it is the standardization planning phase of the concept design.
Setting rules is an important success factor when it comes to in-house production.
If a platform is used recklessly, problems will arise frequently and it will quickly become a negative legacy.
In the article Data Layer and In-Housing, I wrote that the development of a Curated Data Layer and other technologies is important for in-house development.
"Concept design," which is part of our in-house production support services, often has points that go beyond "directly teaching the technology."
- It is also important to teach skills directly.
To lead our customers' digital transformation
As someone who works on the front lines, I believe the following are the skills and knowledge necessary to lead our customers' DX.
- Architectural skills
- Understanding of enterprise architecture
- Ability to draw a roadmap (ability to think about steps)
① and ② are easy to understand, and even when looking at the Digital Skills Standards, they are positioned as skills required for data engineers.
On the other hand, it is difficult to imagine ③.
Thinking like "concept design" is very important.
⇒What is concept design (data integration platform construction service)
Thank you for reading to the end.
The person who wrote the article
