For the Mughal Emperor Babar, one of the strangest characteristics of his newly acquired kingdom in Hindustan was its hydrology. “Autumn crops grow by the downpour of the rains themselves,” he wrote in his memoirs, where he devoted a short section to the description of the strange new land he had entered in the second decade of the 16th century. “[A]nd strange it is that spring crops grow even when no rain falls.” Used to the streams and lakes of the Ferghana Valley in Central Asia, where he was raised, he noted the absence of running water, except in the rivers — “so much so that towns and countries subsist on the water of wells or on such collects in tanks during the rains”.

The young conqueror could only glimpse early in his career how hydrology was central to the organisation of life in the land he had just captured. “In Hindustan hamlets and villages, towns indeed, are depopulated and set up in a moment! If they fix their eyes on a place in which to settle, they need not dig watercourses or construct dams because their crops are all rain grown,” he wrote.

Of course none of this was such a big mystery. In fact, the answer to what puzzled Babar was right in front of his eyes. The presence of wells, for example, was the clearest proof that only 30 feet or 40 feet beneath the ground he was standing on, there was water. If the young invader’s mind had not been so preoccupied with war, he might have realised that there was a connection between rains and wells. After all, it did not take a lot to realise that water from rains recharged the underground aquifers and these were the reason why plants could grow in the spring “even when no rain falls”.

But Babar was a newcomer to India and his views about local hydrology were naïve — at least by Indian standards. The great monsoon rains have been showering their bounties on this land since time immemorial, perhaps even before life emerged in this part of the world. A rich tradition of folklore and religious symbolism has built up around the rains and their arrival. In Gujarat, for instance, since the eighth century at least, the flowering of the Cassia fistula tree has been said to mean that monsoon rains are 45 days away. In parts of Tamil Nadu, a westerly wind in June and July meant rains were expected in the next two months, and farmers procured their seeds and planted their crops accordingly. The manner of flight of a particular bird, the flowering pattern of certain plants or the direction of the wind on important religious occasions like Holi were all used to forecast the arrival of monsoon rains, in times when measurements of sea surface temperatures and atmospheric pressure was unimaginable.

Scientific observation of monsoon only began in the late 19th century, following the failure of rains in 1877 and 1878. Those consecutive years of monsoon failure created the worst ever recorded natural disaster in the world at the time, causing widespread famine and death across India and China. By the time the climatic perturbations behind the failure of the rains ended in 1879, more than 5.5 million people had died due to starvation in India alone. Such was the scale of dependence that life here had developed to the timely arrival of monsoon rains.

Following this disaster, the government of British India set up an observatory to study the Indian monsoon and devise a methodology for predicting its arrival. In his book, The Dance of Air and Sea, Arnold Taylor writes how the first director general of the Indian Meteorology Department, Henry Blanford, “turned to a study of conditions beyond India’s shores,” in his search for the drivers of the great rains, and began compiling data from other territories of the empire. During this exercise, he received a letter from the government astronomer in Australia which contained “the first definitive recognition of an international climatic connection” for monsoon. “Comparing our records with those of India,” the letter read, “I find a close correspondence or similarity of seasons with regard to the prevalence of drought [between India and Australia], and there can be little or no doubt that severe droughts occur as a rule simultaneously over the two countries”.

Blanford himself never made much out of this observation, preferring, instead, to focus his mind on the varying thickness of the Himalayan snows as a predictor of the monsoon rains. In 1904, Gilbert Walker, a 36-year-old statistician from Cambridge, replaced Blanford at the Indian Meteorology Department as director general. He quickly immersed himself in a 15-year study of climate data from around the world. In 1923, he published his findings in which he revealed the operation of a giant “see-saw in atmospheric pressure and rainfall”; he observed a “swaying of pressure on a big scale … between the Pacific Ocean and the Indian Ocean” — when pressure in one place is elevated, it is depressed in the other.

When there was a low pressure system in the Indian Ocean, there would be a dry period on the Pacific side and a wet season on the Indian Ocean and vice versa. Gilbert studied data for the drought of 1877 and 1878 and found “there was a strong pressure reversal over the equatorial Pacific” in those years. Forty-five years after the event, the observatory set up by the colonial authorities had finally explained the devastating drought of those two years.

Gilbert Walker’s study was the first glimpse of the connections that tie the Indian monsoon to climatic phenomena occurring, with a cyclical regularity, on a planetary scale. Walker called it the great Southern Oscillation, and it forms the bedrock of all studies of monsoon rains to this day. Whenever Southern Oscillation reverses, monsoons are affected. For the first time, an answer appeared to be emerging to the question that had hung over the Indian subcontinent ever since life emerged on its grassy plains: How can you tell when monsoon rains are going to come?

But the discovery of Southern Oscillation was not sufficient to answer this question. We knew that a connection existed but to be able to use that knowledge for forecasting purposes, it was necessary to know what drove the oscillation in the first place. In short, it wasn’t enough to know that it existed, we needed to know why it existed. And that discovery was another half century away.

It was in 1957 that the phenomenon known as El Nino was discovered. Taylor writes of how Peruvian fishermen knew, for centuries, of a warm water current that sweeps down along the Pacific coast, each year. The current had been studied in the 19th century, and its effects on marine life described in an article in 1892, by the captain of a boat. His fellow fishermen on those waters “name this countercurrent the current of El Niño (the Child Jesus) because it has been observed to appear after Christmas,” writes the captain.

In 1957, the first observation was made that confirmed El Niño’s effects on the Indian monsoon. The interaction between the ocean currents and the climate was shown to have global ramifications. Whenever an El Niño event appeared off the coast of South America, the monsoons across Asia suffered or failed completely.

But, even as more details started appearing about mechanisms driving monsoon, forecasting its arrival and failure with any meaningful accuracy still remained a distant dream. The main reason for this was logistical. In order to fully observe the El Niño event, it was necessary to take extensive measurements of sea surface temperatures across a large swathe of the Pacific Ocean as well as the Indian Ocean. What is more, this data was required on a regular basis, the more measurements per day, the better. Given that sea surface temperatures were taken using ships, data requirements for forecasting the monsoon were far beyond what the state of technology could deliver at that time.

That began to change in the 1960s. By the middle of that decade, computing models were beginning to be used to process enormous quantities of data that meteorological observations were generating. In 1970, the first satellites equipped with thermal imaging cameras were put into orbit. They were capable of providing high resolution sea surface temperature readings for huge swathes of the world’s oceans.

By the 1990s, advances in computing made possible extremely large and complex models to quickly process enormous volumes of meteorological data. The technique was called ensemble modelling, and saw large numbers of machines working in parallel to do repeated runs on a live stream of meteorological data coming in from a vast network of satellites, weather radars and floating buoys on the ocean surface. Global circulation models were born during this time and a picture of the earth’s climate emerged that was updated on a daily basis.

But technology had only just begun to make climatic phenomena intelligible on a meaningful scale and a picture was slowly emerging of the global circulations of air and water that governed the earth’s climate. At that time, the climate itself began to change, driven by man-made forces that were causing it to morph precisely when it began to yield up its secrets.

In the monsoon-fed regions of the northern subcontinent, a new priority began to compete with the age-old quest to forecast the arrival of rains — flood forecasting. Pakistan had its first three consecutive years of flooding between 1992 and 1995. Bangladesh experienced a flood in 1998 that submerged more than 60 per cent of the country for three months. Both were unusual events. The latter event prompted a search for answers to new questions that the changing monsoon pattern had thrown up: Can the weather system driving the monsoon be predictable on a timescale of months and years in advance? How far can we discern mechanisms that underlie this predictability? Can these mechanisms be modelled? What sort of data observations and transmission systems will be required for “operational prediction” of monsoon-related flooding?

The search for answers to these questions led to the creation of the Tropical Ocean-Global Atmosphere (TOGA) program in 1998. Meteorological scientists, led by Dr Peter Webster, at Georgia Institute of Technology teamed up with people in other research centres to search for mechanisms which linked sea surface temperatures connected with El Niño in the Pacific and unusual monsoon rains in the northern subcontinent. The programme put together the most detailed data on Pacific sea surface temperatures gathered until then, and came to the conclusion that the effects of El Niño, which had guided thinking on monsoon until then, were perhaps overstated. “The picture that has emerged,” wrote the scientists who worked on the programme, “is a system that is global and interactive”. If we are to understand the behaviour of monsoons, particularly for flood forecasting purposes, they said it would be necessary to “extend climate prediction from the Pacific basin to the global domain”.

Starting out as a purely localised phenomenon, by the middle of the 20th century, monsoon came to be understood as part of a large, planetary oscillation linked to the Pacific Ocean. In the opening years of the 21st century, the cutting edge of scientific work discovered that the linkages go beyond that to larger climatic circulations. The data and modelling requirements for flood forecasting, therefore, have become truly stupendous.

It is in this context that a model was developed at the Georgia Institute of Technology in the opening decade of the 21st century. It was capable of ingesting mind-boggling volumes of data from global meteorological databases, and processing them to yield startlingly accurate forecasts of streamflow in the Ganges and Brahmaputra rivers, with a lead time of up to 10 days. The model was deployed in Bangladesh in 2003, and provided accurate forecasts of floods in 2004 and 2007. In 2009, the model was handed over to the Bangladesh government. The creators of the model now turned their attention to Pakistan.

In 2010, Pakistan was struck by the worst floods in its history that displaced close to 20 million people. There have been four monsoon seasons since then, and each one has seen a catastrophic flood caused by unusual rains. The creators of the model arrived just in time.

Five floods in five years is evidence enough that something big is happening around us. So far, Pakistan is lucky that no major breach of a hydrological structure has occurred during any of these floods, but how long will this luck last? Understanding the science behind the torrential rains that fall upon us with biblical ferocity, every year, is critical if we are not to head into a disaster of historic proportions. Developing an action plan to mitigate the impact of floods is now mission critical for Pakistan. We cannot afford to be like the young invader any longer, head addled on war, who could only scratch his head at the hydrological mysteries of India, even as the answers to his questions were right there in front of his eyes.