In this blog post, I would like to highlight the similarities between the physical logistics of moving physical cargo containers and the digital logistics of moving data. In essence, I want to translate the shipping jargon into everyday language to allow you to gain an insight into the world of containerization. Equally for the Container Terminal Managers and their IT teams, this blog post takes optimization from the yard to the server room; I want to highlight that integration of systems is critical and how having the right approach can enable your business to flourish.
The Basics and History of Containerization
The world of container logistics is not something you would commonly come across when you browse the web. However, there is a great book about it by Rose George that describes container logistics as an invisible industry which puts clothes and most of the item’s you see now on the table. Furthermore, you can also learn how the industry transformed from inefficient cargos to solid box containers with a TED-Ed talk by Sir Harold Evans.
The Similarities between Data and Container Logistics
Containers as a “Physical API“
The container is a “physical API” — an interface for interchanging, moving and storing physical things in a standardized, efficient way. MAYA Design describes very elegantly why the Physical API is required and why it is important for the need of “Large, stable, scalable, systems” [Peter Lucas, 2012]. They also conclude how the digital world should consolidate various data interchanges using different “break bulk” protocols (like IMAP, HTTP, POP3) and how the growth in the 21st century will need “Containerization of Information.”
In essence, the efficiency of moving cargo in a regulated box means faster and cheaper movement of goods and therefore, “shrinking the world and enlarging human choice” [Sir Harold Evans]. There are also the added benefits of security by obscurity because if all cargo looked the same, you would not know whether a container is filled with garbage or with the latest smartphones. Someone handling the cargo also does not need to care about what is in it (with a few exceptions such as hazardous cargo).
The biggest benefit is interoperability — being able to move cargo efficiently from A to B agnostically on a large scale on barges, railway wagons or vessels and even on a small scale on forklifts, straddle carriers or RTGs. Being a standard form, it is also possible for the terminal industry to provide services (stripping, stuffing, cleaning, inspections, repairs) around this common factor and shippers or consignees can consume these services consistently and quickly.
The common form factor (described as TEU – Twenty-foot Equivalent Units) is so powerful that there is a large industry created around this form factor in all aspects of management, movement, and optimization.
Fundamental principles of Containerization:
- Standardization– repackage and transform odd sized cargo into well defined and specified containers.
- Movement – once these standardized containers are used, ensure all hand-offs use the standard to improve efficiency, i.e. a common way to move items.
- Hiding Complexity– this allows each part of the chain to deal with their part efficiently.
Data Packets vs. Cargo Containers
In the example of a user posting a photo to Facebook, data moves from a smartphone or a laptop to a Facebook server somewhere across the world.
- The Facebook app will take odd shapes and sizes of data (the photo, text, tags).
- Through some smart packaging (see below: Data Encapsulation and Container Freight Stations), it will send this data over a network.
- The data is then routed via more networks to another machine (a Facebook server), where there may be some orchestration and load balancing.
- The final server then unpacks the packets or consolidates the data and consumes it.
- It is finally posted to your Facebook wall. Below is an illustration of the route data packets take to get to Facebook. You can see they go through quite a journey:
In the containerization world, the ‘packets’ of data are equivalent to a physical box or container which are the same predefined shape and size (typically 20ft or 40ft containers or 1 or 2 TEU). Just how data is encapsulated in a packet, break bulk cargo is encapsulated in a container. This means you may need to stuff a container with multiple cargoes and pallets or split a cargo into multiple containers. With Data Packets it is typical that an ‘item’ (such as a photo) is likely to be split into multiple packets rather than multiple photos being stuffed into a packet; as the packets are significantly smaller than most items transmitted. On the destination, the packets are stitched together again to create the photo initially sent.
A way of thinking about this is having a prefabricated house broken up in a workshop where it was created, transported to its final location in lots of parts and then put back together, so it is the same house again. The pre-fabricated house may be made of multiple workshops in which case there is also need to sequence, combine and in essence orchestrate the components, so they reach the final destination in the right form and sequence.
Below is a map of a shipping container that BBC News followed from London, UK, to California in the US. As you can see, it is a rather unconventional journey:
Data Encapsulation and Container Freight Stations
In data interchanges between applications, data is encapsulated (or packaged) by the sender in logical layers referred to as the Open Systems Interconnection (OSI) layers. In an over- simplification the sender packs the data in these layers and transmits it, while the receiver unpacks the data in reverse order and consumes it.
With container shipping, there is a similar process. For example, you or I may order a new smartphone that ships internationally from Amazon. At the originating location, Amazon (the shipper) will take the individual item, pack it with other items you bought (like a charger, cradle, screen protector), and put it in a single box. This box is likely routed to a container freight station (CFS) or a distribution center; here the oddly shaped boxes are packed based on the destination with other oddly shaped boxes on a pallet. Multiple pallets (potentially from other shippers) are then packed into a container. The container is shipped and then unpacked at the destinated location and is split up into individual packets where they are then routed using local logistics (such as a national parcel service) to your house.
Routing Data and Containers through Networks
In the below scenario the individual items are typically moved from point to point using the shortest route. At each layer, there is a level of abstraction where the detail is buried, and just the relevant information is used to move the item to the next level after a performed activity.
The layers are important as they deal with different logical activities and complexities to be grouped. This means higher layers do not need to be concerned with the complexity; this layered approach has several benefits:
- Hiding Complexity: It means that if (for example) a warehouse shipped your item to a distribution center with 50 other items for other people, the mail carrier who delivers your item to your door need not know the previous activities. Similarly with a web application maybe the data was held in an SQL database are returned 1000 results, when you access the web application it is optimized for your browser on a laptop or a phone displaying the data differently or over multiple pages.
- Ensuring Governance: Let’s assume you ordered something from abroad and there are strict checks done on importing. In the above model, the shipper and customer need not have to co-ordinate the checks. As long as all items go through an available customs checkpoint. For data, this may be an ISP filter or your firewall on your laptop.
- Increasing Flexibility: Finally, let’s assume that warehouse A ran out of the item you ordered, and it needs to come from warehouse B, as long as it gets to the distribution center, once again the mail carrier (or even you) does not need to be concerned with the last minute changes. Once again when accessing a web application, it is likely that there are many servers in multiple data centers. The best one is chosen dynamically based on availability without you or your ISP needing to do anything. Also, over time the provider may move from an on-premise server farm to a cloud one, and you would not even notice.
Containerization – A New Revolution for Traditional Industries
Boxes similar to modern containers had been used for combined rail and horse-drawn transport in England as early as 1792. Modern container shipping celebrated its 50th anniversary in 2006. Almost from the first voyage, use of this method of transport for goods grew steadily and in just five decades, container ships would carry about 60% of the value of goods shipped via sea.
In the last decade pressure on all businesses has increased due to growing competition and external factors; leaders and customers are demanding businesses deliver goods and services better, faster and cheaper. Many of those who have survived have done so by changing their approach by becoming more agile or differentiating themselves. This could be by using improved shipping technology to get goods to customers faster, or often by unlocking digital assets they had all along and better engaging with their customers.
For many businesses, this has meant embracing technology trends to catch up with their competitors. Those who have thrived in this difficult environment have gone one step further and embraced an IT transformation.
Want to Find Out More?
A different approach could unlock your business potential. Take a look at an excellent talk from our CTO, Uri Sarid, of how existing systems can be better used to expose data and drive efficiency and new business growth.
