What Can You Do With A General (Theory of Project Management)?

For those wondering about the origin of the title, it’s (the first part) from an old Bing Crosby song by Irving Berlin.  Blogging has been slow due to travel and some minor travel-related wear and tear.  As Indiana Jones said in the original: “It’s not the years, my dear, it’s the mileage.”

Also related to my slow blogging is in writing longer pieces for AITS.org.  Referring to my last article was the welcome return from a short hiatus by Dave Gordon at The Practicing IT Project Manager blog in general, but his comment on my article “lays the groundwork for a generalized theory of managing software development and acquisition…”  I’m just finishing up the second half of that article now, but Dave identified something that I had probably been working on for quite a while through this observation, though subconsciously: a general theory of project management.

There is no doubt that information plays a large role in PM and one is almost tempted to specify software development and acquisition as a separate area of study.  But I’m not so sure that it merits special handling.  Information theory applies to just about everything in the universe.  As W.T. Grandy from the Department of Physics and Astronomy at the University of Wyoming says in this paper: “In a very real sense the differential equations of physics are simply algorithms for processing the information contained in initial conditions…”  Yet, even a basic definition of information eludes us, and have evolved over time.  Some cosmologists and engineers believe that the universe itself is computational.

Such big ideas are well and fine, but for those of us who must make decisions at our own level of reality we need something more specific.  Our decisions, therefore, could be improved by the development of a general theory of project management.  An interesting proposal along these lines can be found by George Ray for Swiss Management Centre University.  In this case, he posits that the organizing principles of social psychology could be used as the basis for the development of such a theory.  I suspect that a general theory will need to include a multidisciplinary approach to include the mathematical basis derived from the study of complex adaptive systems, the physics that underlies R&D, and social psychology.

Sunday Contemplation — Finding Wisdom — John Steinbeck (Part 2)

later steinbeck


In my last post regarding John Steinbeck, I left off with the great novels of the 1930s, ending with the Grapes of Wrath.  For most novelists his achievements up to this point would be regarded as considerable.  What I am most impressed with is that his own history as a novelist proves how ephemeral such achievements can be.  He was a writer and good one.  His many jobs, especially the occasional newspaper job that he took, seemed to inspire him to his best work.  For more than anything he was a realist.  With his realist eye for detail, and his natural sympathy for people, he enraged both power and privilege through his precise and occasionally remonstrative prose.  I don’t think a better thing can be said of any writer.

With the controversy that followed the publication of The Grapes of Wrath, Steinbeck found solace in spending time collecting marine specimens with his friend and mentor Ed (Doc) Ricketts.  For six weeks the two men rented a Monterey fishing boat with a four man crew named the Western Flyer to travel down the Pacific Coast to Baja California and into the Gulf of California, recording and collecting marine species along the coast.  Ricketts had achieved some success with his book Between Pacific Tides, which became the definitive handbook for the study of intertidal species along the west coast, and which is still considered a seminal work.  The men had planned on co-writing a book about the species found within the tides in the San Francisco Bay area, but the project had come to nothing.  But even Ricketts was eager to get away from his beloved Monterey, suffering from a breakup with a married lover.

The end result of the journey was the book The Sea of Cortez: A Leisurely Journal of Travel and Research (1941), though the trip was anything but leisurely.  The expedition was an exhausting, though fulfilling and happy, one for the men, who concentrated their energies on collecting and cataloguing.  Steinbeck and Ricketts had hoped the sale of the book would at least be sufficient to pay for the expenses of the trip, aside from the sale of specimens to laboratories and public aquaria across the country that they brought back.  It proved to be, however, a commercial disappointment.  Soon the events of the Second World War would overtake any interest in the book.  Furthermore, the notes that underlay the book were those of Ricketts, while the prose to give the notes a narrative structure contributed by Steinbeck.  For the time it was considered an odd book: an uneven read, combining as it did scientific knowledge, storytelling, and contemplations on ecology and humankind’s connection to nature.  With the advantage of time, though, it can be seen that the book made a significant contribution to the science of marine invertebrate identification and distribution along Baja and within the Gulf of California.  The species list, which accompanied the initial editions of the book, is impressive and indicative of the dedication of the two men to their task.  But it also anticipated what later in the century and in our own time has become a common device among science popularizers such as Jacques Cousteau, Stephen Jay Gould, Carl Sagan, Isaac Asimov, and Jared Diamond, among others.  But it goes further than that, for the environmental message in the book anticipates such groundbreaking works as Rachel Carson’s Silent Spring, and Aldo Leopold’s Sand County Almanac.

For Steinbeck, the Second World War would change his life and transform his writing until the end of his life, as it did for Hemingway, Dos Passos, Salinger, Ballard, Heller, Jones, Mailer, Vonnegut, and others.  But unlike the younger writers on that list that would emerge in the post-war period, who could find a new language–oftentimes oblique–to deal with the industrial slaughter of that great catastrophe, the writers from the ’30s–oftentimes also veterans of the Great War–seemed to be struck dumb, horrified by the depths of human cruelty, altering their subjects by finding solace where they could.

A man of action, Steinbeck contributed his writing, at first, to the war effort in the book The Moon Is Down, about a village in an unnamed northern European occupied country (presumed to be Norway) that works to overthrow their invaders through a resistance movement, and Bombs Away: The Story of a Bomber Team.  For the former he received the Norwegian King Haakon VII Freedom Cross.  It was translated into several European languages and distributed secretly to underground organizations across the continent to bolster the morale of similar resistance movements elsewhere, especially in France.  The latter work was done with the cooperation of the U.S. Army Air Corps to increase recruitment.

Not content with continuing work on the home front, he accepted a position with the New York Herald Tribune, traveling with the units in the European theater.  As with his writing during the 1930s, his keen eye for reality informs his stories, telling the story of the life of the common soldier.   He was also recruited as a member of the Office of Strategic Services (OSS), which was the precursor for the CIA.  The actor Douglas Fairbanks Jr. had organized U.S. Navy special warfare units to engage in psychological, diversionary, and deception operations known as the Beach Jumpers.  He participated with Fairbanks in some of these raids, helping to capture a small island off the coast of Italy.  During his service he was wounded several times by shrapnel and suffered from what we now call Post-Traumatic Stress Disorder.

Part Two:  The Pearl through The Winter of Our Discontent

One familiar with Steinbeck’s writing wants to follow him into his sojourn in the reminiscences of his time with Ed Ricketts.  The novels, Cannery Row and Sweet Thursday are guilty pleasures–escapism of the highest caliber.  One cannot visit the town of Monterey without the imprimatur of Ricketts and Steinbeck being noted everywhere, so great is the influence of these works.  Yet they are fantasies–idealized escapes from a tormented man who was reaching back to a simpler time, to the friend and mentor before the fire that engulfed the world.  The man of simple prose recording life as it is abandons reality in these works.  And who can blame him?  He had certainly contributed enough words to what had actually happened, having been there.  He did not possess the desire, as most men of that generation did not, to relive it.  He certainly did not seem to have the vocabulary to transform it into fiction.  Hemingway took a similar course with his writings of Cuba, Africa, anti-heroes running nightclubs or fishing boats, finally recapturing part of his voice with The Old Man and the Sea.

For Steinbeck the book that combined the return of his voice with his desire to look to the past was through the novella The Pearl (1947).  As with many of his stories, it is based on a folk tale, in this case one that he heard when visiting the region in Baja California in 1940 with Ricketts.  In the story, the child Coyotito is stung by a scorpion.  The parents, Kino and Juana, must find a way to pay for treatment for their young son.  Kino, who is a pearl diver, finds an enormous pearl, which everyone in the village covets.  They find that the local pearl auction is rigged, the buyers attempting to convince Kino that what he found is worthless.  Soon misfortune follows the family as the villagers and trackers attempt to take the pearl from them.  They try to make their way to the capital, where the auctions are not rigged, but are met with tragedy along the way.

The Pearl operates on many levels.  In high school it is often required reading, and most teachers present it as a parable of human greed, materialism, and the actual value of things.  But Steinbeck is not so simple.  For the pearl also represents anything of value that an individual may possess, whether it be tangible or intangible.  It is also something taken from nature, which Kino is convinced by the society in which he lives can be turned into money.  Its beauty disappears the longer it is out of its element so that by the end of the story it is a grotesque object.  Remembering Steinbeck’s influences, when people are separated from their humanity great misfortune follows.  For the village of the pearl its very presence corrupts everything around them, blinding them from acknowledging the humanity, the connections that bind them together as human beings.  As with his earlier stories, great misfortune results, and usually it falls on the most vulnerable.

Recovering from his depression from the war, the breakup of his marriage, and the death of Ed Ricketts, we find what Steinbeck intended to be his magnum opus, East of Eden (1952).  On the surface this is a novel about the Hamilton and Trask family in the Salinas Valley of California.  In reality, though, the novel moves away from the story of the Samuel Hamilton, the family patriarch, who Steinbeck modeled on his maternal grandfather, and toward the Trasks.  This is because the Hamiltons become the bedrock of the Salinas Valley, representing stability and good, as opposed to the Trasks, who want the same thing that binds the Hamiltons together, but which alludes them because of their poor decisions, despite the great wealth (though possibly misbegotten) that bought them the best farm in the valley.

It is also this characteristic that makes the Trasks more interesting.  Steinbeck apparently saw this himself as he transforms the second half of the novel into a parable based on the biblical account of Cain and Abel.  Adam Trask, who has had a difficult growing up back east, inherits money from his father’s estate, though he suspects that the fortune was dishonestly obtained.  He takes pity on Cathy Ames, who seems to be the victim of violence, and marries her, not knowing that she is cruel and a murderess.  Having deluded himself into domestic bliss, Adam soon finds out about Cathy’s true character.  She shoots and wounds him after giving birth to twin boys, Caleb and Aron, in making her escape from the boredom of domestic life.  Finding her way to the town of Salinas, she changes her name and takes over as the madam of the most notorious whorehouse in the county.

Adam tries to raise his boys with the help of his Cantonese cook, Lee, and the Hamiltons on the adjacent farm.  Adam is inspired to copy the success of Samuel Hamilton, but loses the family fortune in a badly conceived business venture.  As he nears maturity Caleb, the troubled son, is determined to redeem his father’s shame at losing the family fortune and goes into farming himself.  Aron, the “good” son, decides to attend Stanford and become an Episcopal priest.  The brothers vie for the affections of the beautiful daughter named Abra from one of the most well-to-do families in the valley.

Moody and always testing limits, Caleb discovers that his mother, who his father had said had died, was the notorious Madam Kate in the town of Salinas.  He keeps this knowledge a secret from both his father and his brother, but it fuels his skepticism of both his father’s sanctimoniousness, and his brother’s goodness.  Soon the First World War breaks out and Caleb enters into a business venture with Will Hamilton, the son of Samuel, to sell beans from the valley to European buyers at a premium.  He is wildly successful at this scheme and cannot wait to present the money to his father, seeking his father’s love and approval which always seemed to allude him.

When Aron returns from school, hoping not to be upstaged by his brother, Caleb presents the money to his father.  Adam rejects the money, characterizing it as tainted, compared to the pure motives of his brother.  In retaliation Caleb brings Aron to meet their mother, destroying the boy’s illusions.  The ties between father and sons, and between brothers, are now cut (the ties between the mother and her children long since severed), and in typical Steinbeck fashion tragedy ensues, though there is a sort of redemption at the end.

I am of two minds regarding East of Eden.  On the one hand it is an impressive work.  It introduces narrative elements that were extremely unconventional in 1952–anticipating the devices of changing narrative perspectives, and describing the times and places with his usual precision.  On the other hand, however, there is a heavy-handedness to the writing.  He takes the biblical story of Cain and Abel and turns it into a black-and-white assessment of human frailty.  The Trasks are doomed from the start, and one cannot but resent the author for making them so.  As such he not only diminishes the complexity of the human condition, but also dilutes the themes that he had explored in his previous works.  Still, it is a great novel and much wisdom can be found here, for in Steinbeck’s telling Caleb (as Cain) is the son most driven by that most human of all human needs–the need for love–while Aron (as Abel) is driven only by the slender reed of societal respectability.

For most literary critics Steinbeck’s best work usually ends here with East of Eden, with perhaps a mention of the non-fiction book Travels with Charley (1962).  In retrospect, however, while I find Steinbeck’s observations recorded in that volume regarding the changes that were overtaking his beloved California and the racial hatred he witnessed in the American South both interesting and as clear-eyed as usual, I keep coming back to The Winter of Our Discontent (1961) as his last significant work.

The book introduces us to the character Ethan Allen Hawley, whose family once was among the aristocrats of the seaside village of New Baytown somewhere in New England.  Now the Hawleys are common folk, Ethan having to make a living as a grocery clerk.  With so many reminders of the family’s once great past among the artifacts of their sprawling ancestral home, his wife Mary, and their children, Allen and Ellen, are ashamed of their lack of resources.  His friends criticize him for his integrity, suggesting that he take bribes, or be more ruthless in his business dealings, especially with his boss, Alfio Marullo.

Succumbing to the pressure to improve his economic condition he finds out that Marullo is an undocumented immigrant and in the country illegally.  Ethan turns Marullo in to the Immigration and Naturalization Service, and the man is deported.  Before deportation, however, Marullo turns over the store to Ethan because of all of his years of his honest service, believing him deserving.  Having achieved this misbegotten gain, Ethan then seeks to take control of a strip of land owned by his best friend Danny, who is the town drunk, and on which the town plans to build an airfield.  Ethan gives Danny money to get treatment for his alcoholism in exchange for willing the land to him.  Danny, instead of seeking treatment, slips the will under the door of the store and is soon found dead with a bottle of whiskey and sleeping pills.

Thus, almost overnight Ethan has achieved the success that his wife and friends always wished for him.  Soon, however, he is in for a shock as he learns that his son has won an essay contest by plagiarizing the books found in their old home library.  When he confronts his son the boy is not remorseful, and it is this knowledge that wakes him to the corruption inherent in his own actions.  It is at this point that Ethan realizes that he must do something to atone for his greed.

As with so many of Steinbeck’s books, he was ahead of his time with The Winter of Our Discontent.  Some hailed it as his best work since The Grapes of Wrath, but most judged it an inferior work, preachy and cynical.  Given the time–the optimistic years of the early 1960s–Steinbeck’s novel seemed to be an unnecessary downer, misplaced in an era of expanding opportunities.  But with time it has been observed that he identified a sickness in the American character that was soon to overtake the nation in the ensuring years.  As such, the novel explores the same issues explored by Fitzgerald in The Great Gatsby and Dreiser in An American Tragedy.  What is missing in Winter is the shocking dénouement that characterized his best work of the 1930s, and which is present in both Gatsby and American Tragedy.

In summary, John Steinbeck observed and recorded both the best and worst of the human character.  The wisdom in his books concern the same issues with which we grapple today–how to separate what is important from the material, how to stay true to our natures, and in staying true to our natures how to adhere to the best part of our natures.  He saw people for what they were and wrote of them sympathetically and accurately, even when they behaved badly.  As such we find that expecting to find perfection in the human species is a silly game and a fool’s quest.  We can do nothing but what is right and behave humanely–that the choice is in our hands–and that love is the organizing principle of our species.

Saturday Night Music — Remembering B. B. King

There have already been a number of on-line tributes to B. B. King.  I prefer to remember him for his music and the way he made his guitar “Lucille” sing for him.

I came to the blues during two periods of my life.  The first was as a youth in the mid-1960s when bluesmen like Muddy Waters, Lightnin’ Hopkins, John Lee Hooker, Howlin’ Wolf, B. B. King, and others were “rediscovered” during the British music invasion.  Then in the early 1980s a blues revival hit the country with the introduction of the CD, placing the credit (and many financial rewards finally) where it belonged.  Suddenly the older recordings of Son House, Robert Johnson, Charlie Patton, Sonny Boy Williamson, and others became available side-by-side with the electric sound of Robert Collins, Buddy Guy, Robert Cray, and Stevie Ray Vaughn, among so many others.

The blues have spoken to me ever since.  It is the music of a people who face hard times and indignity, yet are able to enjoy the little things in life.  As an American folk form, it is among the headwaters of jazz, rhythm & blues, and rock and roll.  Without the blues, the full expression of the human condition cannot be documented.  Unlike so-called “Classical” music, which is really a set of disparate styles of music from different periods, the blues does not exist or continue to exist because of the social and cultural ambitions of political and economic elites.  Instead, it is the music of the people: it originally springs from a people–from the freed-slave African-American experience–who were denied equal rights, their humanity, basic dignity, and the ability to express themselves when they wished in the normal course of life until the last quarter of the 20th century.  But the blues, sung in roadhouses, honky tonks, juke joints, front porches, and within the safety of their homes, gave life to this expression.

As any American should or would, I found a common appeal to the blues, though I come from a different set of ethnic origins, part of the immigrant experience in fleeing Europe.  For within the blues is expressed the basic striving of humanity.  My background growing up in my earliest years in the tenements of Hoboken, a melting pot of immigrants and ethnic groups all striving to one day find a better life and some human dignity, is, I think, the reason for it.  There I viewed and played with other children of every color and hue.  Even later, during our own flight into the suburbs, that experience from my formative years stayed with me.

I listened to an interview that B. B. King gave back in the early ’90s where he explained that in the blues there is always some twist that demonstrates the comedy in the tragedy of human life.  He cited as his example the song “Nobody Love Me But My Mother.”  The lyrics go like this: “Nobody loves me but my mother.  And she could be jiving too.”

Now, brother, that is the blues.

But the blues doesn’t discriminate, because as with all American folk music, it a democratic music.  Women blues singers, both black and white, reach back to Bessie Smith, Ma Rainey, Victoria Spivey, Billie Holiday, and Big Maybelle, through Alberta Hunter, Etta James, Koko Taylor, Sippie Wallace, Nina Simone, to more recent blues singers such as Irma Thomas, Bonnie Raitt, Janis Joplin, Susan Tedeschi, Shemekia Copeland, Marcia Ball, and Ruth Brown, and the list could go on.  The blues is alive and vital.

The blues as a folk form originated from a different place, but intersected with the folk music of singers as varied as Woody Guthrie, Pete Seeger, the Carter Family, Hank Williams, and Jimmie Rodgers.  Sometimes both traditions became embodied in one person, such as Leadbelly (Huddie Ledbetter) and the aforementioned Muddy Waters.

Like the music and the society around him, B. B. King changed with the times.  His early music concentrated on rhythm & blues and electric blues focused on a largely African American audience.  Later in life he expanded the blues audience into the larger society, merging it with other musical styles such as blues rock.  As such, he became the penultimate blues popularist.  So in tribute to the great B.B. King, who will be missed, I bring you two of his performances on the excellent Austin City Limits, one from 1983 and one from 1996.  The King is dead, long live the blues.

Gotta Serve Somebody — The Proper Balance of Duties in Business–and Project–Management

While traveling over the last couple of weeks I was struck by this article in the Wall Street Journal entitled: “Pharmaceutical Companies Buy Rivals’ Drugs, Then Jack Up the Prices.”  The reporter of the article stated in a somewhat matter-of-fact manner that the reason for this behavior was the need for maximization of stockholder value.  Aside from the fact that, with the poorly vetted excuse mongering in the article about fewer opportunities for development and limitations on payments under healthcare, U.S. drugs tend to be significantly higher than generics found overseas, the assumption regarding maximizing stockholder value is misplaced.

As Steven Pearlstein pointed out in this Wonkblog piece back in 2013, maximizing shareholder’s value is not part of the fiduciary duties of the company nor its CEO.  Such a view is not supported in either tradition or law.  A corporation, as any Business 101 student will tell you, is an artificial person established for the purposes defined in its charter.  This charter is issued by the sovereign–in our case the sovereign consists of the citizens of the states that issue corporate charters through their representative governments.

As Pearlstein rightly points out:  “The fiduciary duty, in fact, is owed simply to the corporation, which is owned by no one, just as you and I are owned by no one — we are all “persons” in the eyes of the law. Shareholders, however, have a contractual claim to the “residual value” of the corporation once all its other obligations have been satisfied — and even then directors are given wide latitude to make whatever use of that residual value they choose, as long they’re not stealing it for themselves.”

The obligations of a company include not only shareholders but also customers, suppliers, and employees and others with whom the corporation establishes contractual relations.  What Pearlstein’s article also shows is that companies that take the position that customer and employee satisfaction comes first are those that maintain and grow market share for a sustained period of time.  This should be no surprise.  Short term management yields either short term results or total failure.  This instability due to the shift in marking for the market for stockholder value was reflected in the 2007-09 Great Recession, where companies with little ability to weather the storm found themselves on the rocks and shoals, and with angry customers and employees left in the lurch.

My question for my colleagues is always this:  are you in it for the long or short term?  If I get the latter answer then I say good luck to you, but we steer this ship on a different course.

The ramifications for project management probably make this conflict between the push for stockholder primacy against other co-equal interests more salient.  To the project manager, the primacy of the project is defined by its contractual line items that are designed to meet customer requirements and expectations.  Stockholders don’t deliver products and services.  Those who wish to extract resources prior to those needed to satisfy customer requirements and contractual commitments–and pay employees and subcontractors in those efforts–whether they know it or not, are, at best, engaging in bad business practices, and, at worst, are engaging in fraud.

Once again, this ship steers a different course.  Staying customer-focused is what has and will continue to win out at the end of the day.

Over at AITS.org — Maxwell’s Demon: Planning for Obsolescence in Acquisitions

I’ve posted another article at AITS.org’s Blogging Alliance, this one dealing with the issue of software obsolescence and the acquisition strategy that applies given what we know about the nature of software.  I also throw in a little background on information theory and the physical limitations of software as we now know it (virtually none).  As a result, we require a great deal of agility inserted into our acquisition systems for new technologies.  I’ll have a follow up article over there that provides specifics on acquisition planning and strategies.  Random thoughts on various related topics will also appear here.  Blogging has been sporadic of late due to op-tempo but I’ll try to keep things interesting and more frequent.

Let’s Get Physical — Pondering the Physics of Big Data

I’ll have a longer and less wonky article on this and related topics next week at AITS.org’s Blogging Alliance, but Big Data has been a hot topic of late.  It also concerns the business line in which I engage and so it is time to sweep away a lot of the foolishness concerning it: what it can do, its value, and its limitations.

As a primer a useful commentary on the ethical uses of Big Data was published today at Salon.com in an excerpt from Jacob Silverman’s book, Terms of Service: Social Media and the Price of Constant Connection.  Silverman takes a different approach from the one that I outline in my article, but he tackles the economics of new media that were identified years ago by Brad DeLong and A. Michael Froomkin back in the late 1990s and first decade of the 21st century.  This article on First Monday from 2000 regarding speculative microeconomics emerging from new media nicely summarizes their thesis.  Silverman rejects reforming the system in economic terms, entering the same ethical terrain on personal data collection that was explored by Rebecca Skloot on the medical profession’s genetic collection and use of tissue during biopsies in the book, The Immortal Life of Henrietta Lacks.

What Silverman’s book does make clear–and which is essential in understanding the issue–is that not all big data is the same.  To our brute force machines data is data absent the means of software to distinguish it, since they are not yet conscious in the manner that would pass a Turing test.  Even with software such machines still cannot pass such a test, though I personally believe that strong AI is inevitable.

Thus, there is Big Data that is swept up–often without deliberate consent by the originator of the data–from the larger pool of society at large by commercial companies that have established themselves as surveillance “statelets” in gathering data from business transactions, social media preferences, and other electronic means.

And there is data that is deliberately stored and, oftentimes shared, among conscious actors for a specific purpose.  These actors are often government agencies, corporations, and related organizations that cooperatively share business information from their internal processes and systems for the purpose of developing predictive systems toward a useful public purpose, oftentimes engaged in joint enterprises toward the development of public goods and services.  It is in this latter domain that I operate.  I like to call this “small” Big Data, since we operate in what can realistically be characterized as closed systems.

Data and computing has a physical and mathematical basis.  For anyone who has studied the history of computing (or has coded) this is a self-evident fact.  But for the larger community of users it appears–especially if one listens to the hype of our industry–that the sky is the limit.  But perhaps that is a good comparison after all, for anyone who has flown in a plane knows that the sky does indeed have limits.  To fly requires a knowledge of gravity, the atmosphere, lift, turbulence, aerodynamics, and propulsion, among other disciplines and sciences.  All of these have their underpinnings in physics and mathematics.

The equation that we use in computing is known as Landauer’s Principle.  It is as follows:

kT In 2,

where k is the Boltzmann constant, T is the temperature of the circuit in Kelvins, and In 2 is the natural logarithm of 2.

This equation follows those in thermodynamics established earlier in physics.  What this means is that the inherent entropy in a system–its onward inevitable journey toward a state of disorder–cannot be reduced, it can only be expelled from the system. For Landauer, who worked at IBM in physical computing, entropy is expelled in the form of heat and energy.  For the longest time, given the close correlation and applied proofs of the Principle, this was seen as a physical law, but modern computing seems to be undermining the manner in which entropy is expelled.

Big Data runs up against the physics identified in Landauer’s Principle because heat and energy are not the only ways to expel entropy.  For really Big Data entropy is expelled by the iron law of Boltzmann’s Constant: the calculation of probable states of disorder in the system. The larger the system, the larger the probable states of disorder, and the more our results in processing such information become a function of probability.  This may or many not matter, depending on the fidelity of the probabilistic methods and their application.

For “small” Big Data, the acceptability of variations from the likely outcome is much narrower.  We need to approach being 100% correct, 100% of the time, though small variations are acceptable depending on the type of system.  So, for example, in project management systems, we can be a percent or two off on rolling up data, since accountability is not an issue.  Financial systems compliance is a different matter.

In “small” Big Data, entropy can be expelled by pre-processing the data in the form of effort expended toward standardization, normalization, and rationalization.  Our equation, kT In 2, is the lower bound, that is, it identifies the minimum state of entropy that need be expelled in order to process a bit.  In reality we will never reach this lower bound, but we can approach it until the difference between the lower bound of entropy and the “cost” of processing data is vanishingly small.  Once we have expelled entropy by limiting the states of instability in the data, expelling the cost of entropy through the data pipeline, we can then process the data to derive its significant with a high degree of confidence.

But this is only the start.  For once “small” Big Data undergoes a process to ensure its fidelity, the same pattern recognition algorithms used in Big Data can be applied, but to more powerful and credible effect.  Early warning “signatures” of project performance can be collected and applied to provide decision-makers with information early enough to affect the outcome of efforts before risk is fully manifested, with the calculated probabilities of cost, schedule, and technical impacts possessing a higher level of certainty.


Sunday Music — KT Tunstall performing “Tangled Up in Blue” and “Invisible Empire”

Talk is that she is in L.A. to pursue a movie career, movie soundtracks, and work on her next album.  For those of you unfamiliar, KT Tunstall hails from St. Andrews, Fife in Scotland.  She hit the music scene in 2004 and took it by storm.  Every album seems to get better than the last.

Here she is on an early performance from the U.K. on the iconic John Dylan cover which became for her a signature song for awhile–let’s hear it for the dustbin!

Before heading to L.A. for a change of career in 2014, she released the excellent Invisible Empire//Crescent Moon album.  Here she is with a live version of the title song from the first half of the album.

Her musical and emotional growth are apparent simply in the juxtaposition of the videos and we are the richer for it: documenting insights for all of us to ponder as we navigate existence.  Let’s hope we hear something again soon from KT.