What you need to know about containerization and “the internet of physical things”

binary code containerization

In this blog post, I would like to highlight the similarities between the physical logistics of moving physical cargo containers and the digital logistics of moving data. In essence, I want to translate the shipping jargon into everyday language to allow you to gain an insight into the world of . Equally for the Container Terminal Managers and their IT teams, this blog post takes optimization from the yard to the server room; I want to highlight that integration of systems is critical and how having the right approach can enable your business to flourish.

The Basics and History of Containerization

The world of container logistics is not something you would commonly come across when you browse the web. However, there is a great book about it by Rose George that describes container logistics as an invisible industry which puts clothes and most of the item’s you see now on the table. Furthermore, you can also learn how the industry transformed from inefficient cargos to solid box containers with a TED-Ed talk by Sir Harold Evans.

The Similarities between Data and Container Logistics

Containers as a “Physical API

The container is a “physical API” — an interface for interchanging, moving and storing physical things in a standardized, efficient way. MAYA Design describes very elegantly why the Physical API is required and why it is important for the need of “Large, stable, scalable, systems” [Peter Lucas, 2012]. They also conclude how the digital world should consolidate various data interchanges using different “break bulk” protocols (like IMAP, HTTP, POP3) and how the growth in the 21st century will need “Containerization of Information.”

In essence, the efficiency of moving cargo in a regulated box means faster and cheaper movement of goods and therefore, “shrinking the world and enlarging human choice” [Sir Harold Evans]. There are also the added benefits of security by obscurity because if all cargo looked the same, you would not know whether a container is filled with garbage or with the latest smartphones. Someone handling the cargo also does not need to care about what is in it (with a few exceptions such as hazardous cargo).

The biggest benefit is interoperability — being able to move cargo efficiently from A to B agnostically on a large scale on barges, railway wagons or vessels and even on a small scale on forklifts, straddle carriers or RTGs. Being a standard form, it is also possible for the terminal industry to provide services (stripping, stuffing, cleaning, inspections, repairs) around this common factor and shippers or consignees can consume these services consistently and quickly.

The common form factor (described as TEU – Twenty-foot Equivalent Units) is so powerful that there is a large industry created around this form factor in all aspects of management, movement, and optimization.

Fundamental principles of Containerization:

  • Standardization– repackage and transform odd sized cargo into well defined and specified containers.
  • Movement – once these standardized containers are used, ensure all hand-offs use the standard to improve efficiency, i.e. a common way to move items.
  • Hiding Complexity– this allows each part of the chain to deal with their part efficiently.

Data Packets vs. Cargo Containers

In the example of a user posting a photo to Facebook, data moves from a smartphone or a laptop to a Facebook server somewhere across the world.

  1. The Facebook app will take odd shapes and sizes of data (the photo, text, tags).
  2. Through some smart packaging (see below: Data Encapsulation and Container Freight Stations), it will send this data over a network.
  3. The data is then routed via more networks to another machine (a Facebook server), where there may be some orchestration and load balancing.
  4. The final server then unpacks the packets or consolidates the data and consumes it.
  5. It is finally posted to your Facebook wall. Below is an illustration of the route data packets take to get to Facebook. You can see they go through quite a journey:

containerization map

In the containerization world, the ‘packets’ of data are equivalent to a physical box or container which are the same predefined shape and size (typically 20ft or 40ft containers or 1 or 2 TEU). Just how data is encapsulated in a packet, break bulk cargo is encapsulated in a container. This means you may need to stuff a container with multiple cargoes and pallets or split a cargo into multiple containers. With Data Packets it is typical that an ‘item’ (such as a photo) is likely to be split into multiple packets rather than multiple photos being stuffed into a packet; as the packets are significantly smaller than most items transmitted. On the destination, the packets are stitched together again to create the photo initially sent.

A way of thinking about this is having a prefabricated house broken up in a workshop where it was created, transported to its final location in lots of parts and then put back together, so it is the same house again. The pre-fabricated house may be made of multiple workshops in which case there is also need to sequence, combine and in essence orchestrate the components, so they reach the final destination in the right form and sequence.

Below is a map of a shipping container that BBC News followed from London, UK, to California in the US. As you can see, it is a rather unconventional journey:

shipping map

Data Encapsulation and Container Freight Stations

In data interchanges between applications, data is encapsulated (or packaged) by the sender in logical layers referred to as the Open Systems Interconnection (OSI) layers. In an over- simplification the sender packs the data in these layers and transmits it, while the receiver unpacks the data in reverse order and consumes it.

With container shipping, there is a similar process. For example, you or I may order a new smartphone that ships internationally from Amazon. At the originating location, Amazon (the shipper) will take the individual item, pack it with other items you bought (like a charger, cradle, screen protector), and put it in a single box.  This box is likely routed to a container freight station (CFS) or a distribution center;  here the oddly shaped boxes are packed based on the destination with other oddly shaped boxes on a pallet. Multiple pallets (potentially from other shippers) are then packed into a container. The container is shipped and then unpacked at the destinated location and is split up into individual packets where they are then routed using local logistics (such as a national parcel service) to your house.

routing data and containers through networks

Routing Data and Containers through Networks

In the below scenario the individual items are typically moved from point to point using the shortest route. At each layer, there is a level of abstraction where the detail is buried, and just the relevant information is used to move the item to the next level after a performed activity.

layered transmission

The layers are important as they deal with different logical activities and complexities to be grouped. This means higher layers do not need to be concerned with the complexity; this layered approach has several benefits:

  • Hiding Complexity: It means that if (for example) a warehouse shipped your item to a distribution center with 50 other items for other people, the mail carrier who delivers your item to your door need not know the previous activities. Similarly with a web application maybe the data was held in an SQL database are returned 1000 results, when you access the web application it is optimized for your browser on a laptop or a phone displaying the data differently or over multiple pages.
  • Ensuring Governance:  Let’s assume you ordered something from abroad and there are strict checks done on importing. In the above model, the shipper and customer need not have to co-ordinate the checks. As long as all items go through an available customs checkpoint. For data, this may be an ISP filter or your firewall on your laptop.
  • Increasing Flexibility: Finally, let’s assume that warehouse A ran out of the item you ordered, and it needs to come from warehouse B, as long as it gets to the distribution center, once again the mail carrier (or even you) does not need to be concerned with the last minute changes. Once again when accessing a web application, it is likely that there are many servers in multiple data centers. The best one is chosen dynamically based on availability without you or your ISP needing to do anything. Also, over time the provider may move from an on-premise server farm to a cloud one, and you would not even notice.

Containerization – A New Revolution for Traditional Industries

Boxes similar to modern containers had been used for combined rail and horse-drawn transport in England as early as 1792. Modern container shipping celebrated its 50th anniversary in 2006. Almost from the first voyage, use of this method of transport for goods grew steadily and in just five decades, container ships would carry about 60% of the value of goods shipped via sea.

In the last decade pressure on all businesses has increased due to growing competition and external factors; leaders and customers are demanding businesses deliver goods and services better, faster and cheaper. Many of those who have survived have done so by changing their approach by becoming more agile or differentiating themselves. This could be by using improved shipping technology to get goods to customers faster, or often by unlocking digital assets they had all along and better engaging with their customers.

For many businesses, this has meant embracing technology trends to catch up with their competitors. Those who have thrived in this difficult environment have gone one step further and embraced an IT transformation.

Want to Find Out More?

A different approach could unlock your business potential. Take a look at an excellent talk from our CTO, Uri Sarid, of how existing systems can be better used to expose data and drive efficiency and new business growth.


We'd love to hear your opinion on this post

2 Responses to “What you need to know about containerization and “the internet of physical things””

  1. Hi Sanket! The similarities noted bring to light an exceptional way of viewing both data and container logistics. A great read. That said, I would suggest possibly a second look this time with a broader scope. While I like to consider myself an optimist, not everything in container logistics operates smoothly. I would encourage you to broaden this blog and look at the similarities from both sides of the coin – meaning the difficulties. Often times we need to identify and evaluate what goes wrong, to architect a positive way forward. There is little doubt that containerization has introduced great benefits to the world, so my comment is not meant to be negative in any way. I am merely suggesting we need to also consider the negative effects of actions such as funneling, content integrity and risk mitigation. Containers are funneled through ports at which point they are subject to 3rd party delays like Customs inspections, added cost like demurrage, and more. In similar fashion, data is funneled through service providers – internet, cloud, etc., and subject to spam filters, speed limitations, and other actions; equally causing delays and other negative impacts to data transmission. Cargo consolidation places varied goods from multiple users into a single container, exposing all contents to contamination derived from any of the other items; effecting content integrity. Similarly, data packets expose all encapsulated data to any virus a single piece of data may have. In mitigating the risk of smuggling a weapon of mass destruction, containers are subjected to profiling, radiation portals and other security measures, effecting both the path and duration of container travel. In similar fashion, data is often screened with virus protection applications, denied access by firewalls, and other cyber defense solutions. These are only a few issues that cause a negative impact to operations – both data and container logistics, the effect of each must be considered when considering process optimization fostering efficiency and new business growth.

    • Hi Michael, some really interesting observation (around ‘funneling’) and I think they are valid and of a concern in the maritime sector today.

      A former customer of mine in South Africa has several terminals across the country and several of them close to each other. They employ dynamic berth scheduling across some of the more closer terminals to do some load balancing and also to route vessels to terminals that are available to handle the discharge of containers; of course some additional handling of nearby export containers are required but the overhead is outweighed. This really helps them to reducing anchorage times and increase berth utilization. I would say this pattern is similar in technology terms as clustering which have similar benefits.

      Another customer in the north of the UK similarly employs a port strategy called ‘Port Centric’ this is to deal with some of the landside bottlenecks that were common over the last few decades with the ‘centralized distribution centers’ in essence rather than taking all cargo to the center of the country it is stripped and stuffed (separated and aggregated) in warehouses in the port area; then shipped to the final destination. This avoids the unnecessary trips where cargo is sent to the center of the country just to be transported back closer to the original port. This also helps to remove road congestion, pollution and transportation costs. This distributed model removes the central distribution center bottlenecks.

      There are a few other patterns being employed by ports globally to address some of the issues you mentioned. However, these types of changes (unlike software) can’t be employed by downloading the latest patch and often take a long time to implement and subsequently optimize.

      Needless to say both data logistics and container logistics share common patterns and issues. There is a lot of learning in both directions in what works and what doesn’t. Luckily in software we can employ new patterns quickly and adapt to the changing landscape.