HULFT Development Story | HULFT Usage Column Special Contribution
Released in 1993, purely Japanese file sharing software
The story behind HULFT development
HULFT is file sharing middleware that was released in 1993. We are publishing columns written by its creator, Shinji Nishikawa, about the development of HULFT from the 1980s to the 1995s.
*Affiliation, title, company name, etc. are as of the time of the interview.
Episode 1 begins
Let me start by telling you how I first became interested in packaged products.
In the 1980s, I was in the Technology Development Department, where I was involved in developing tools to improve developer productivity. Later, I was told that I should gain experience in business systems, so I was put in charge of the accounts receivable management system for Seibu Department Store. For six months, I completed the system by catching the last train home almost every day, but even after it went live, I was exhausted by the constant problems. The lesson I learned there was that "business systems require a lot of effort and little benefit." It was around this time that I began to think that it might be better to provide general-purpose products to a large number of people.
In the late 1980s, end-user computing began to spread. PCs began to be used in business departments other than information systems. In the 1990s, signs of the Internet's spread became apparent, and the era shifted from a time when protocols differed from manufacturer to standardized TCP/IP. It was also around this time that "Neodama," or "networks, open systems, downsizing, multi-vendor and multi-field," became a trend in the computer industry.
Amid these changes, our company was forced to respond to open systems. However, since there was not a single person familiar with open system technology, we purchased UNIX for research purposes and began a process of trial and error. It was at this point that the company ordered my department, which was then the manager of the Technology Development Department, to "create a new packaged product."
"HULFT" stands for "Host Unix Link File Transfer"
The idea for HULFT came about when my subordinate Itano said, "Distributed systems are going to be popular in the future!" Until then, information search databases had mainly been based on mainframes, but mainframes were expensive and inefficient. Therefore, we thought that in the future, systems would be downsized to open systems.
The original concept was to "create a copy of the mainframe database in an open database," in other words, to realize the now-common practice of "virtualization."The idea was to create a system that would "enter search criteria into a database, determine which database the data is in, and automatically search it."
However, building a "virtualized database" on the machines of the time was very difficult. Ultimately, the first version of HULFT was downgraded to a more realistic approach. Our company had originally dealt with many file transfer mechanisms in our order-taking and ordering systems. Applying this, we created a system that automatically linked to the database via file transfer.
It took six months to conceive the idea and six months to develop it, with a team of five people, including Itano, Sato, Kobayashi, and myself. The functionality was simple, with only one-way communication from the mainframe to UNIX.
"HULFT" is an abbreviation for "Host Unix Link File Transfer." There were many name candidates, but I liked the sound of "Haru." The season of "spring" has a bright feeling, and it is also reminiscent of the artificially intelligent computer "HAL9000" that appears in "2001: A Space Odyssey." However, we were worried that a name that was too similar might not be able to be trademarked, so we combined the initials of "File Transfer" and came up with the name "HUL-FT." And thus, HULFT was born.
Episode 2: Signs of Growth
The opportunity to upgrade HULFT soon came. It was a request from Arthur Andersen (then), who was consulting on Seibu Department Store's POS system. They needed a function that could file transfer via TCP/IP and collect send data in both directions (initially, HULFT only communicated one way, from the mainframe to UNIX).
At the time, our company did not have any C language engineers. Therefore, five new employees were assigned to work on HULFT upgrade. Immediately after joining the company, they were sent to external training for six months, where they were thoroughly educated on UNIX and the C language from the basics. The members included Ishibashi, Tanabe, Maeda, and others.
Unfortunately, development of this POS system was discontinued midway, and HULFT 2 was never adopted.
The first user of HULFT 2 was Credit Saison.
I recall that it was rare at the time for mainframes to be able to communicate via TCP/IP. As a result, we gradually began to receive inquiries from companies that had noticed the convenience of HULFT. At that time, we received a request to create a system that would be compatible with Fujitsu's mainframe OS, XSP. We responded immediately to business opportunities. To achieve this, our mainframe team spent about six months porting the system.
We also began developing a Windows NT version to expand our sales channels. However, we did not have any engineers capable of working on the Windows NT version. We assigned five people to work on porting it from the UNIX version, including Tanaka, who had just returned from a secondment at Fujitsu, and Takei, Ito, and Igarashi, who were in their second year at the company. At the time, our approach was to have young employees learn new technologies from scratch and expand the range of compatible models.
During this period, we actively expanded the range of UNIX operating systems available, provided there was customer demand. We devised coding methods to make porting to other models easier, and we even managed to port a system to another model in just two weeks.
Customers chose HULFT for its superior functionality
However, sales were still sluggish at this time. Although Tamada, Nakazono, Hatakeyama and others were expanding sales, there was no sales network other than direct sales, and the product had only been introduced to around 20 companies.
Although it is now said to hold approximately 80% of file transfer market (according to a 2017 survey by Fuji Chimera Research Institute), at the time there was a considerable amount of competition and the competition was tough.
This is a bit of a backstory, but at some point, our sales staff started to talk about how they couldn't win competitions. HULFT had more features, but we couldn't compete on price. So, at the behest of then-President Shibayama, who was determined to develop HULFT into a marketable product, we decided to create a budget version with limited functionality called HULFT Dash. Not a single copy of this product sold.
This was because, as long as we won the competition, the customer would choose HULFT, which has superior functionality, over HULFT Dash in subsequent business negotiations.
HULFT was growing into a product that customers chose.
Episode 3: Extensions
Yokogawa Electric Corporation approached us about an MVS (IBM) version of HULFT. They really wanted to transfer data between mainframes and UNIX. However, at the time, our company did not have an IBM mainframe, so we decided that development would be difficult. Instead, we proposed that they develop it themselves, and we would make the source code public.
However, the TCP/IP APIs for Fujitsu mainframes and IBM mainframes were significantly different, so development seemed to be quite difficult. With our cooperation, we were able to expand the IBM version.
Yokogawa Infotec (at the time) also wanted to sell our products, so they became our first distributor.
Soon after, a director at Yokogawa Electric advised us that if we were to sell to external parties, we should have a full range of general-purpose machines from domestic manufacturers.
This led to the development of the Hitachi version. However, Hitachi headquarters already had a distribution send package. This made it difficult to develop it at headquarters, so they introduced Hitachi Information Network (then an information processing subsidiary). They made the IBM version source code public and asked Hitachi Information Network to develop the Hitachi version, which led to a distributor agreement with what is now Hitachi Information Systems (then).
We believed that the future of this product would depend on how well it could be differentiated.
HULFT 3 was created with an eye on the competition, aiming to improve functionality and performance. While HULFT 2's two-way send was limited to between mainframes and UNIX, the two-way functions, such as kanji conversion, which had become a problem between different models, were revised, and a Windows version was also released, making it possible to transfer data between most open models on the market at the time, as well as between mainframes. Centralized monitoring by Manager was also introduced, and a significant number of functions were added that would appeal to customers.
Around the time HULFT 3 was released, a company whose main business is clinical testing such as blood testing asked us to create an OS/400 version as they were planning to install 200 OS/400 units.
However, since there was no one in the company with experience with OS/400, it was decided to send Taniuchi, Tsuchiya, and the other three for external training. However, at the time, there was no one in the world who was creating software for OS/400 by combining TCP/IP and C language. The training was limited to teaching them how to operate OS/400 and how to compile C language. After that, they had no choice but to proceed by learning on their own, feeling their way around. Development equipment was also borrowed, and the project was completed in about 10 months.
We believed that the future of this product would depend on how well it could be made compatible with multiple vendors and how well it could be differentiated. That's why we worked with the determination to "meet customer requests."
Everyone was desperate to develop HULFT into a marketable product.
Episode 4: Happy Scream
Perhaps due to word of mouth among hardware manufacturers, the company's name recognition grew, and requests for development began pouring in from a variety of users.
One customer requested that we connect 200 Fujitsu K Series store servers to TANDEM Nonstop machines, while another company requested that we use HULFT to connect their factory K Series servers to a mainframe.
The K Series did not have the C programming language, so development was done in COBOL. COBOL had problems when developing general-purpose programs, such as the inability to dynamically change record length. Development was requested from Fujitsu Kansai Systems (then a manufacturer) that had the technology.
The TANDEM version was initially requested by a customer who wanted to transmit data between Tandem Computers' (then Tandem Computers) TANDEM, which was used at FEP. Later, when TANDEM was equipped with UNIX-based functions, Tandem Computers requested that we port the UNIX version. However, since we did not have the know-how regarding NonStop or a computer for development (TANDEM was expensive anyway), we proposed that we make the specifications public and create it.
At first, we thought we would have to learn the technology in-house, but the manufacturers, Fujitsu and Tandem Computers, decided to build their own in-house systems to support the development. Tandem Computers in particular offered to handle post-development upgrades and sales, which sped up development. It was around this time that we gradually began to realize the high demand for HULFT.
At the time, HULFT 's development team consisted of fewer than 15 people.
Version 3 of HULFT was released in 1994, around the time when the use of UNIX machines as server computers was rapidly accelerating. Although it was after the collapse of the bubble economy, technological innovation was progressing and corporate IT investments were continuing to increase.
There were many computer manufacturers, each thriving and releasing their own unique UNIX machines: NEC's EWS, Fujitsu's DS, Hitachi's HIUX, DEC's Alpha, NCR's SVR4... It was not uncommon for a single machine to cost tens of millions of yen, but users began to flock to these diverse computers.
In this environment, customers who had already implemented HULFT on their general-purpose machines began to introduce new UNIX machines. Naturally, they wanted to connect to these machines using HULFT. We began to receive many requests to support models that HULFT did not yet support.
The development team saw this customer request as an opportunity and worked tirelessly every day to move forward with the porting.
The number of deliveries also increases dramatically.
At the time, developers would burn software onto CD-Rs on their own computers, affix labels, box them up, and ship them, but this was no longer enough, so it was around this time that we introduced a dedicated CD-R burner and established a team dedicated to shipping.
At the time, HULFT 's development team consisted of fewer than 15 people. Although there was constant overtime and busy days, the team was able to create an environment where they could always take on new challenges, and to foster a strong team.
Episode 5: And the De Fact
In 1995, the introduction of Windows 95 sparked a surge in internet usage.
It was thought that anyone with a computer, modem, and regular telephone line would be able to communicate data, and that this would accelerate data exchange between companies.
At that time, business-to-business communications were based on the BSC procedure developed by IBM in the 1960s, and were not intended for use over the Internet. The two most common protocols were the "Zengin Procedure" recommended by the Japanese Bankers Association and the "JCA Procedure" established by the Japan Chain Stores Association (JCA). These protocols were also widely used in our clients' systems.
HULFT 's functionality surpassed the communication environments based on these old procedures, and customers began to express their desire to use HULFT in conjunction with the Internet for inter-company communications. If HULFT could be used instead of the somewhat outdated "JCA procedure," Japan's inter-company communications environment would improve dramatically! With this in mind, Itano and Tamada visited the distribution industry associations of the time.
"Could you please use HULFT as an alternative to the JCA procedure?"
An answer I never expected came back.
"Feel free to make something like this!"
The organization also wanted to create its own infrastructure, but as a closed process to the distribution industry.
This, in turn, ignited a fire in the HULFT unit.
With version 4, I finally felt like "It's selling!"
The main concept behind the next HULFT was to make it available over the Internet and create a standard communication procedure that could be used across industries. This version included many of the functions that are now commonplace, such as firewall compatibility and CSV conversion functionality for EDI.
Around this time, the company also began to strengthen its sales promotion activities for HULFT. HULFT The logo color was a challenging red, and the company actively placed advertisements in Nikkei Computer and other IT magazines published by Nikkei Business Publications, Inc.
We also paid particular attention to the boxes (packaging) that the media and manuals would be placed in when shipped. We commissioned the design to Mitsuo Miyahara, a designer who was active in Italy at the time, and from among several designs, everyone involved in development, sales, and sales promotion decided on one that represented the image of HULFT. Although it was an unusual design, it naturally became a catalyst for everyone involved to develop a strong attachment to HULFT.
For the first time, our company held a product launch event at the Tokyo Prince Hotel, inviting partners and users, and thus "HULFT 4" was released to the world in 1996. It was also around this time that we began to use a logo that integrated the version number into HULFT.
A product will not be successful unless it has the determination to commercialize it and, once the product is complete, not only development, but also sales promotion, sales, and support function well together like the four wheels of a car. This version gave us a different feeling. We received a steady stream of inquiries from all industries. It has even begun to be applied in the banking industry, which had previously been a high barrier to entry. With version 4, we finally felt that "it's selling!"
By 1998, six years after sales began, cumulative shipments had reached 10,000 units and 700 companies.
HULFT has finally grown into a product that can be branded as the "industry standard." This is how the idea that "it would be better to provide a general-purpose product to many people" was realized.
