More on Excel…the contributing factor of poor Project Management apps

Some early comments via e-mails on my post on why Excel is not a PM tool raised the issue that I was being way too hard on IT shops and letting application providers off the hook.  The asymmetry was certainly not the intention (at least not consciously).

When approaching an organization seeking process and technology improvement, oftentimes the condition of using Excel is what we in the technology/PM industry conveniently call “workarounds.”  Ostensibly these workarounds are temporary measures to address a strategic or intrinsic organizational need that will eventually be addressed by a more cohesive software solution.  In all too many cases, however, the workaround turns out to be semi-permanent.

A case in point in basic project management concerns Work Authorizations Documents (WADs) and Baseline Change Requests (BCRs).  Throughout entire industries who use the most advanced scheduling applications, resource management applications, and–where necessary–earned value “engines,” the modus operandi to address WADs and BCRs is to either use Excel or to write a custom app in FoxPro or using Access.  This is fine as a “workaround” as long as you remember to set up the systems and procedures necessary to keep the logs updated, and then have in place a procedure to update the systems of record appropriately.  Needless to say, errors do creep in and in very dynamic environments it is difficult to ensure that these systems are in alignment, and so a labor-intensive feedback system must also be introduced.

This is the type of issue that software technology was designed to solve.  Instead, software has fenced off the “hard’ operations so that digitized manual solutions, oftentimes hidden from plain view from the team by the physical technological constraint of the computer (PC, laptop, etc.), are used.  This is barely a step above what we did before the introduction of digitization:  post the project plan, milestone achievements, and performance on a VIDS/MAF board that surrounded the PM control office, which ensured that every member of the team could see the role and progress of the project.  Under that system no one hoarded information, it militated against single points of failure, and ensured that disconnects were immediately addressed since visibility ensured accountability.

In many ways we have lost the ability to recreate the PM control office in digitized form.  Part of the reason resides in the 20th century organization of development and production into divisions of labor.  In project management, the specialization of disciplines organized themselves around particular functions: estimating and planning, schedule management, cost management, risk management, resource management, logistics, systems engineering, operational requirements, and financial management, among others.  Software was developed to address each of these areas with clear lines of demarcation drawn that approximated the points of separation among the disciplines.  What the software manufacturers forgot (or never knew) was that the PMO is the organizing entity and it is an interdisciplinary team.

To return to our example: WADs and BCRs; a survey of the leading planning and scheduling applications shows that while their marketing literature addresses baselines and baseline changes (and not all of them address even this basic function), they still do not understand complex project management.  There is a difference between resources assigned to a time-phased network schedule and the resources planned against technical achievement related to the work breakdown structure (WBS).  Given proper integration they should align.  In most cases they do not.  This is why most scheduling application manufacturers who claim to measure earned value, do not.  Their models assume that the expended resources align with the plan to date, in lieu of volume-based measurement.  Further, eventually understanding this concept does not produce a digitized solution, since an understanding of the other specific elements of program control is necessary.

For example, projects are initiated either through internal work authorizations in response to a market need, or based on the requirements of a contract.  Depending on the mix of competencies required to perform the work financial elements such as labor rates, overhead, G&A, allowable margin (depending on contract type), etc. will apply–what is euphemistically called “complex rates.”  An organization may need to manage multiple rate sets based on the types of efforts undertaken, with a many-to-many relationship between rate sets and projects/subprojects.

Once again, the task of establishing the proper relationships at the appropriate level is necessary.  This will then affect the timing of WAD initiation, and will have a direct bearing on the BCR approval process, given that it is heavily influenced by “what-if?” analysis against resource, labor, and financial availability and accountability (a complicated process in itself).  Thus the schedule network is not the only element affected, nor the overarching one, given the assessed impact on cost, technical achievement, and qualitative external risk.

These are but two examples of sub-optimization due to deficiencies in project management applications.  The response–and in my opinion a lazy one (or one based on the fact that oftentimes software companies know nothing of their customers’ operations)–has been to develop the alternative euphemism for “workaround”: best of breed.  Oftentimes this is simply a means of collecting revenue for a function that is missing from the core application.  It is the software equivalent of division of labor: each piece of software performs functions relating to specific disciplines and where there are gaps these are filled by niche solutions or Excel.  What this approach does not do is meet the requirements of the PMO control office, since it perpetuates application “swim lanes,” with the multidisciplinary requirements of project management relegated to manual interfaces and application data reconciliation.  It also pushes–and therefore magnifies–risk at the senior level of the project management team, effectively defeating organizational fail safes designed to reduce risk through, among other methods, delegation of responsibility to technical teams, and project planning and execution constructed around short duration/work-focused activities.  It also reduces productivity, information credibility, and unnecessarily increases cost–the exact opposite of the rationale used for investing in software technology.

It is time for this practice to end.  Technologies exist today to remove application “swim lanes” and address the multidisciplinary needs of successful project management.  Excel isn’t the answer; cross-application data access, proper data integration, and data processing into user-directed intelligence, properly aggregated and distributed based on role and optimum need to know, is.

Finding Wisdom — Four novels by Nadine Gordimer

Nadine Gordimer passed away this year on July 14, 2014.  Rarely is there an opportunity for a gifted writer to be both brave and essential.  She was both of these throughout her 90 years.  Gordimer, of course, was a South African novelist and short story writer.  Her fiction dealt with the issues regarding race and the racial apartheid that defined South African society during a time when writing and openly speaking about such issues was forbidden.  She addressed these issues when similar regimes of apartheid, white supremacy, and racial segregation were in force elsewhere, most notably in the American South and in Rhodesia, thus her words spoke to millions beyond the borders of her native country, where many of her books were banned.  But she went beyond just writing, placing herself in jeopardy by joining the African National Congress (ANC) in support of overturning apartheid during a time when membership in that organization was illegal, participating in anti-apartheid demonstrations, and hiding ANC members in her home from the police who were sought for arrest.  She was close friends with the attorneys of Nelson Mandela’s defense team during his trial, helping him prepare his “I Am Prepared to Die” speech, which he gave during his defense in 1964.  Years later, after his release from prison, Gordimer was one of the first people Mandela sought out.

Gordimer’s fiction explores the society around her in a progression of discovery that I suspect very much traces her own intellectual and emotional progression.  She had begun writing when she was 15 years old, a largely isolated only-child of nurturing and protective parents.  By the time she was married for the first time and had her first child in her mid-twenties, she submitted one of her stories to The New Yorker and was published there for the first time in 1951.  Additional stories–which she continued to believe was the most essential fictional form for her time–and novels ensued; quite a number of them.

In her first novel, The Lying Days, which she published in 1953, we follow the growing awareness of a twenty-four year old woman by the name of Helen Shaw to the realities of both apartheid and the small town life–with its other prejudices and taboos–in which she lives.  It is in this novel that Gordmer’s keen eye for the essential truth of a matter and her ability to communicate it in her fiction was first revealed.  The vultures of South Africa hover everywhere, she wrote, over both the veld and the cities, and in doing so they look down on all of the people of the plain and the cities, both the rich and the poor.  In the city of Johannesburg and its outskirts where poverty tends to collect, as in all cities, there is that thing called charity.  But, she wrote, ”in South Africa there is one difference, a difference so great that the whole conception of charity must be changed.  The people…were not the normal human wastage of a big industrial city but…the entire black-skinned population on whose labor the city rested…too poor to maintain themselves decently because no matter what their energy, their skill, their labor was not allowed value above subsistence level.”  But if this were her only insight it would be slight indeed, but Gordimer plumbs the society around her with a keen eye for detail: the brute labor and hopelessness in working in the local mine, the Jewish boy who dare not talk of his identity or declare his love for Helen, the black girl with whom Helen befriends in university who cannot come to Helen’s home, and who would be turned away in any event by Helen’s parents.  Helen’s lover as young woman, Paul, works to provide some measure of human kindness to the poor of the city and is frustrated at every turn.  Through the eyes of Helen and the other characters in the novel we see a panorama of the conflicts and frustration that makes up South African society under its strictures and oppressive taboos regarding race, ethnicity, and religion.  From the outside–and now with the benefit of history–we can see that this is a regime that cannot hold.  But beyond a novel of ideas, the greatest sin that Gordimer committed as a member of that society in this first novel is what is essential to making a great writer, it is in humanizing her characters and bringing them forth as three dimensional, communicating to the reader that these people in their interactions have an internal life like our own, regardless of their skin color or their background.  To those conservatives and defenders of the social order in her own time, this was just the first of many sins should would commit.

In the story Occasion for Loving, which was published in 1963, is told by the observant third party.  In this novel Jessie and Tom Stilwell are part of the liberal intellectual class of South Africa, leading a comfortable suburban existence.  It is through Jessie that the story is told.  The Stilwells supplement their income by renting out extra space and in this case it is to newly married Ann and Boaz Davis.  Boaz is a composer but he has been suffering writer’s block and so is busy collecting and transcribing the native tribal music of Africa before it disappears.  He is also Jewish in a land hostile to Jews.  Ann, for her part, is open to experience and challenges convention at every turn.  She is a young English woman used to getting her way.  Into the picture enters Gideon Shibalo, a talented and passionate young African man who has received a fellowship to study painting in Rome, but who is denied a passport by the South African authorities.  The Stilwells and Davises feel for Gideon’s injustice.  The Stilwells, in particular, given their status live in a world where normal social convention doesn’t seem to touch them. They travel to the townships at will and have contact with the Africans in defiance of the authorities, supporting organizations to overturn apartheid.  Along the way Ann falls in with Gideon.  But this is not some esoteric societal transgression.  The relationship between Ann and Gideon is a political crime with serious consequences if they are found out.  It is hard to tell Ann’s motivation when she and her lover decide to run away together, whether her personal “occasion for loving” is due to her true feelings or some other high minded motivation to save the young man’s dreams.  And so it is with all of the characters and the tragedy which this story becomes.  They try to insulate themselves from their actions through intellectualizing their actions, refusing to see that “…Every contact with whites was touched with intimacy; for even the most casual belonged by definition to the conspiracy against keeping apart.”  In the end Ann is convinced by Jessie, who faces her own conflicts regarding her former first marriage, the child from that marriage, and her three children with Tom, to end the affair and return to Boaz.  In the end she is a realist regarding the difference between familial love and sexual love.  But Ms. Gordimer is not going to let her characters off the hook that easily, lest the book become another trivial potboiler.  She uses this form to explore the other aspects of the characters, the affair, and the larger context in which they occur.  Jesse may be a realist in matters of the heart, but can she really understand the motivation for the basic freedoms that Gideon is denied?  In the story’s “occasion for loving,” how complicit are the Stilwells and Davises, for all of their liberalism and moderation, in the oppressive and racist system that was South Africa?

A Guest of Honour published in 1970 garnered her the James Tait Black Memorial Prize, which was her first major literary award.  It tells the story of a British colonial administrator by the name of Evelyn James Bray.  Mr. Bray is a pariah among the white settlers of the colony that he serves because of his activism in favor of the black freedom movement, and is forced to flee as a result.  A while after these events, however, the colony is granted its independence and Mr. Bray is invited back to the new republic by its chosen president.  The president is Adamson Mwete, a popular and gregarious man who lives in poverty one day only to be propelled to the head of a new country the next.  His closest friend and advisor has been Edward Shinza, both an intellectual man and one who can turn thought into action.  The two men–Mwete and Shinza–completed one another, were opposite sides of the same coin, but when Bray returns he notices that Shinza has fled to the bush.  Bray, who initially takes a passive role in celebrating the newly independent state, is pulled into the events that now take on a life of their own.  For the differing visions of Mwete and Shinza play out across the country in real time.  Mwete believes that the state first and foremost must benefit in order for everyone to benefit, while Shinza sees the benefits of freedom needing to play out in practicality, changing the lives of the people for the better.  Mwete is the more adept politician and so overwhelms his old friend and ally.  In implementing his program, he outsources the country’s mines to foreign interests, with the state sharing in the profits.  The hitch is that the workers must accept lower wages and poor working conditions.  The unions are thus co-opted to enforce the will of the state, as is the former independence party apparatus.  The workers, who had been the vanguard of the independence movement, are suppressed.  As popular discontent grows Mwete enforces ever increasingly oppressive measures, creating a police state not so different from the one that existed prior to independence, tying himself closer to the empire from which the country fled.  Foreign interests are invited in to bolster the regime and the rich are allowed to keep their fortunes and rule over the wage earners, subsistence farmers, and the poor; tied as they are to the largesse of the state and foreign economic interests.  Soon the old revolution returns, the president flees to England, and foreign troops arrive to restore order.  The storyline in A Guest of Honour seems all too familiar today with the benefit of 44 years hindsight since its publication; similar stories having been played out across Africa and Asia.  It is the story of a revolution gone bad, of ideals betrayed to expediency, of greed, of human stupidity and ignorance borne of the desire to do good but without the tools or the knowledge to know how to go about it.  It is an indictment of paternalism, of colonialism, of economic imperialism, and the savage cruelty of the strong over the weak.  For Gordimer writing from the perspective of 1970, it is a warning–a cautionary tale–of how things could turn out in the wake of apartheid’s removal.  But it is not simply a didactic exploration of philosophies or politics or consequences.  The characters live and breath and–all too frequently–err, as those in all great literature do.

Gordimer’s 1974 novel The Conservationist won her the Booker Prize, the most prestigious literary award in England, the equivalent to the National Book Award in this country.  To many it is considered her masterpiece, though there are many candidates for that title.  The novel’s lyrical telling is much like an impressionist painting, allowing the reader to see details that are only faintly described, the colors and overall effect communicating more than the misleading simplicity of the subject matter.  The main character is a wealthy white industrialist from Johannesburg by the name of Mehring, and it is through his perspective that the story is told.  He buys a 400 acre farm less than an hour from his work as a meeting place for his mistress, Antonia, and because the losses from the farm’s operations is a tax write-off.  In his mind he loves the land, but treats it as any other investment, viewing the productiveness of the cattle and cornfields as the ultimate measure of his stewardship while, at the same time, dismissing the concerns and well-being of the Zulu caretakers who run the farm.  The same can be said for all of the people in his life–he is disconnected from them and sees them only in terms of his holdings or what they offer him, in particular the need for young women to feed his sexual appetite.  The farm’s foreman, Jacobus, finds a dead body on the farm.  The police are called but the deceased man is black, and so the circumstances of his death are of little official concern.  The police bury the body where it is found.  This knowledge haunts Mehring throughout the novel.  The story, of course, is allegory, but one that contains a great deal of psychological wisdom and human insight.  It deals with the immediate issue of apartheid but it reveals much about human nature.  In the mind of Mehring we find a man whose self-image is driven by wanting to be seen as doing the “right” thing, of being a “proper” human being (one cannot characterize the self-interest he seeks and which is his central defining characteristic as “good”), at least in his mind’s eye.  That the language and perceptions of a racist and materialist worldview color his perspectives does not in the least come to mind.  For all of his wealth and internal drive there is little self-reflection or self-awareness.  It is only when the body of the unidentified black man is exhumed by a flood and he witnesses the black farm hands burying the man as if a relative that he feels his own isolation.  But that is the condition of all who would be rulers of a kingdom, even the petty ones of our own times.

In her overall body of work, for which she received the Nobel Prize in 1991, Nadine Gordimer challenged in her writing not only apartheid but all forms of oppression.  Her books are both cleared-eyed and brutally honest.  The wisdom to be learned from her body of work sits not in the polemics of freedom, but into the insights of how people come to terms with a great evil.

There are other works that I could have chose aside from these four.  There are her works that were famously banned in her native country:   A World of Strangers (1958)which tells the story of a white South African man who witnesses the brutality of apartheid and is force by conscience to join an organization like the ANC.  The novella, The Late Bourgeois World (1966), which was banned for a decade, directly attacks the privileged white suburban life upon which the slavery and repression of the black majority is based.  Burger’s Daughter (1979) about the daughter of a Communist activist in South Africa in the wake of her father’s death.  July’s People (1981) banned during the apartheid and post-apartheid period, in which she imagines a black revolution turned oppressive against white people, an upside down apartheid.

There are also the more recent novels.  A Sport of Nature (1987) about an angry young woman without a cause who is caught up in the politics of South Africa without being emotionally touched by them,  The House Gun (1998) explores the psychology of seemingly reasonable people who are forced to face the reality of their lives from a single act of violence.  The Pickup (2001), about the challenges of two lovers from different cultures without a country.  No Time Like the Present (2012), which chronicles the struggle of life in South Africa after the struggle.

In all of Ms. Gordimer’s works there are connections that tie people together even under a system of forced separation, though the psychological barriers of separation are just as real.  In the end, no matter what kind of justifications are built to separate people or that people use to insulate themselves or their tribe or their identify, the fact is that we are all connected in some way for what happens in the world around us.  Her writing attacks prejudice wherever it tries to hide, whether it be in others or in ourselves.

In thinking about the significant body of work left by Nadine Gordimer–for the short stories and short story collections, which I haven’t addressed here, are significant–I am struck by the fact that the American South never produced an author of the same stature in dealing with the defining evils of segregated southern society.  Certainly no one that combined Gordimer’s bravery, conviction, and writing talent.  Instead, we are left with only the alcohol-infused paternalist voice of William Faulkner, who dealt with issues of “miscegenation” early, but the oppression he witnessed is chronicled only obliquely, writing directly about what went on as a matter of course only once:  in his excellent Intruder in the Dust.  There are fairy tale stories and domestic concerns of Eudora Welty, the southern gothic of Carson McCullers, and the apologists like Robert Penn Warren (later reformed),  and Elizabeth Spencer.  Harper Lee, a southern expatriate, gave us To Kill a Mockingbird and nothing else.

This is not an indictment, necessarily, of American southern literature.  Certainly the effects of Jim Crow and the Black Codes have been told by African American authors (James Baldwin, Ralph Ellison, and others) and the occasional works of white southerners (such as William Styron) to great effect–and there are certainly other aspects of living in the American south.  But I find it interesting that the one voice during a significant period in our own history that consistently spoke against racial prejudice and oppression and the blind spot that societies construct to mask its effects and beneficiaries–and which appeared regularly in publications like The New Yorker–came from a South African author.  For this we owe a great deal of thanks to Nadine Gordimer.

Saturday Music Interlude — Three from London Grammar

London Grammar consists of vocalist Hannah Reid, guitarist Dan Rothman, and multi-instrumentalist Dot Major.  Out of university from Nottingham, U.K., Reid and Rothman met in the dorms and began recording and posting their music on-line.  “Hey Now” became an internet viral hit in 2012 and they released a debut album in 2013.

Their music is dominated by the emotive voice of Reid, whose powerful vocal instrument always stays in the foreground but is supported and propelled effectively by both Rothman and Major.  The three acting in empathy effectively mine the emotional potential of each song.

The following two videos from WFUV showcase the band’s raw talent in an organic, unproduced setting.  Their version of Chris Isaak’s classic “Wicked Game” performed here is a their performance evokes both the sorrowful reflection and obsessive passion expressed in the song’s lyrics, though it is apparent that “Hey Now,” a self-penned song, possesses more urgency and passion in the minds and hearts of the artists.

Finally, here is their most recent official video which most effectively displays the band’s potential backed by full instrumentation and with Reid’s disciplined but powerful vocals tested to their full range.

Doctor My Eyes — Excel is Not a Project Management Tool (and neither is PowerPoint)

This is not to disparage the utility of a good spreadsheet to take care of those transient requirements to take a bit of data from the reporting systems and to run some custom algorithms or trends to perform what-if or other one-off analysis.  Probably most of us do this occasionally.

What I am referring to is the condition in many organizations in which data that consists of information essential to business operations is kept and analyzed using spreadsheets or other flat delimited storage or text methods.  The issue here is the optimum use of information, which the use of Excel and PowerPoint does not achieve.  Before anyone thinks that this is a contrarian’s post that is critical of Microsoft products, one need only read the technical advantages of true relational database management systems that are managed by specialized language like MS SQL.  Each of these applications and products has their proper place.

I would prefer that this post would be unnecessary since the title should be considered self-evident at this point–it is very similar to a presentation I gave almost 20 years ago when still advising senior managers in the U.S. Department of Defense and the aerospace industry–but the facts tell us that it’s assertion not so self-evident.  For example, a survey of 262 financial executives by the Financial Executives Research Foundation and the staffing firm Robert Half in 2012 indicated that 64% of public and private companies in the U.S. still used spreadsheets and manual methods for their financial solutions.  In 2011 a study by the ERP company IFS found that “of more than 281 manufacturing executives, 75 percent of study respondents aged 35 and under report using desktop spreadsheet software like Microsoft Excel instead of their company’s ERP, customer relationship management (CRM), supply chain management (SCM) or other enterprise software” while “respondents aged 36 to 45, 58 percent said they would use desktop spreadsheet software instead of their company’s designated enterprise applications.”

There are certainly good reasons in each study that people sub-optimize in using their data.  First and foremost is the hassle and expense of the centralized IT office.  The same methods and organizations that secure data and are tasked with maintaining configuration control for the IT resources of an organization are the same ones that oftentimes tend to impede flexibility in the acquisition of needed digitized business process solutions.  Back in the 1980s as the PC began to displace the old mainframes run by the stodgy folks wearing white shirts, pen holders, and black glasses who had to be bribed, cajoled, and stroked in order to obtain processing time, access to archived data, or–heaven forbid–devise and run a program, a new world order was declared in which the centralized and non-responsive central IT office was declared dead forever.  Then digital crime raised its ugly head along with a cacophony of new products, some of which delivered on their promises, but most which did not.  Thus the need, once again, to bring some semblance of order to the chaos that incompatible and ineffective products (among other problems) wrought.  So IT has come full circle and we are back to highly bureaucratized IT organizations (or aggressive IT services companies) that in many cases will defend their turf ruthlessly, many times to the detriment of the interests of the organization.  Thus, people do what they have always done when inflexible rules get in the way–they find ways around them.  The most convenient way is to use the “legal” workaround, which is the spreadsheet, the Word document, or the PowerPoint presentation.  Along with being able to achieve what is needed without having to ask permission or miss a deadline, managers and employees oftentimes learn these basic applications first.  They are intuitive, readily available, and familiar.

The problem, of course, takes many forms.  The first is that manual methods are rife with errors.  In fact, study after study shows that almost 90% of spreadsheets contain errors.  In 2007 CIO Magazine published an article listing the eight worst spreadsheet errors of all time, highlighting the very real damage that relying on these methods have created to organizations and businesses.  And this was before the financial crisis revealed similarly large errors in the financial markets during the most recent housing bubble deflation and resulting Lesser Depression.

In providing project management solutions to my target verticals, the area most in need of remediation–and which presents the most immediate opportunity for return on investment, improving data and process credibility, and preventing fatal business errors–is in displacing those processes built around managing data using Excel, Word, and PowerPoint.  In identifying data that is most appropriate for moving to a relational database management solution, I have found that this data shares certain characteristics:

a.  The data is an element essential to decision-making and must therefore be credible.

b.  The data is an element essential to trend analysis, organizational history, or corporate knowledge.

c.  The methods and algorithms used in the data’s processing must be provide results that are repeatable and consistent.

d.  The various elements of data are interrelated and may originate from systems of record.

g.  The output from the processing of the data is essential to job of more than one person or one process.

h.  The time it takes to construct, maintain, update, adjust, and use the manual processing environment for the data greatly exceeds the marginal value of the effort that could be saved using more automated methods to achieve the same or greater results.  That is, replacing the manual and spreadsheet method of using the data results in greater productivity and either cost savings or work shifting to more productive tasks tied to project success.

As a localized tool or as an intermediate point of review for data residing in tables augmented by more robust automation, Excel has proven itself to be a workhorse.  For in-depth papers and extended communication Word is the appropriate medium and for presentations PowerPoint provides a good basis for simplified communication of an idea or set of ideas.  But for the processing and integration of enterprise data these applications are inappropriate–and can be quite damaging–to business operations.

Saturday Music Interlude — Tony Bennett and Lady Gaga in Brussels and in the Studio

There is not much more that can be said about Tony Bennett.  He is a living treasure.  Since “I Left My Heart in San Francisco” I have followed him on his musical odyssey and then later, as a man in my 30s, explored the rich musical legacy of his earlier years.  It is hard to believe that Tony Benedetto of Astoria, New York, is 88 years old.  The surprise for me on the duets these artists have performed is the wonderful instrument that is the voice of Lady Gaga (Stefani Joanne Angelina Germanotta of Yonkers, New York).  It is almost as if her musical career was destined to lead her to this album.  Her powerful voice is both expressive and enveloping, tuned perfectly to the tempo and the register of the music.  I am only disappointed that there is no video of “Lush Life” performed by Lady GaGa solo which critics have described as one of the best renditions of this difficult song.  This collaboration, which began life from a benefit concert in New York, plows no new ground for these jazz standards.  But big things always start out with small steps.  Lady Gaga has always been known for her ability to handle vocal gymnastics and jazz music is a gold mine of challenging songs that would suit her voice and attitude.




Family Affair — Part II — The Micro and Managerial Economics of Projects under Public Monopsony

In my last post I summarized by the macroeconomic environment in which we operate and delved into some discussion of microeconomic foundations.  The response was positive if lukewarm overall, but ego-boosting is not why I started a blog.  One of my readers once asked why I don’t take on some hot button issues.  Well that’s not my role or area of expertise.  I’m not a politician or a social commentator.  The community I inhabit has a large impact but is relatively small and mostly consists of engineers, scientists, mathematicians, some policy-makers, thought leaders, and other technically-focused professionals.  I’m not trying to stir up emotions.  I’m out to stimulate discussion and thought.  I’m relieved that I don’t get trolls when posting factual information that goes against popular misconceptions.  They are a waste of time.

So for me this blog serves two purposes.  First, it is a public discussion of sorts on topics both familiar and unfamiliar, but which I consider interesting.  I do a lot of background research for each post and usually learn something new.  That’s why I don’t post every day.  I give things some thought and research what I write about.  I think it useful to share these discoveries and insights.  The ones that relate to my professional development and expertise in software and project management get posted to my company’s website.

Secondly, I think we live in interesting times.  This is meant both positively and ironically.  I think it useful to record the human condition as we find it in our daily lives during the period of human history we inhabit, and to record many of the intellectual and cultural issues that are topical.  I always find it interesting to return to a subject years later through the writings of others as well as my own to see how they stack up against the perspective of time.  Dr. Roger Spiller of the Army’s Combat Studies Institute, one of my mentors for my graduate degree in history, once told me that if I can look back at anything I’ve written and not be embarrassed by something in those writings would make me a very lucky man.  I’m not that lucky.  But taking the chance to be wrong is part of the price of intellectual curiosity.  Recognizing and admitting one’s wrongness is part of intellectual honesty and maturity–and we are nothing if not thinking.  Our intellect is the most significant feature that we possess that distinguishes us from the other animals.  It is in the cliche’, the ideology, the spin, the doctrine–types of non-thought–where evil finds comfort.

So in continuing from my last post; in understanding the basis of the macroeconomic environment in which we operate, how these systems behave, and how the microeconomics under the macroeconomic environment are affected by the framing of the macro rules of behavior allow us to then flesh out the microeconomics and managerial economics of project management.

As I stated in my last post on the topic, markets are really good at establishing price and pretty bad mechanism for anything else.  Everyone who has taken basic economics understands the basic supply and demand curve.  Here is an example for housing*:


So when we make decisions for projects within an organization or firm regarding price, which then allows us to determine the resources we will have to do business given a set of assumptions on possible sales volume, it is useful to understand the nature of the market in which it inhabits, and the competitive environment of that market.  Some examples follow.

Public Monopsony

I have a great deal of experience in operating in this type of market from my perspectives as a business operator and owner and, previously, as a government Contracting Officer and acquisition professional.  Monopsony exists where there is one dominant buyer of the supplies and services from that market.  There are both public and private monopsony (and some are forced by unfair competitive practices such as exclusivity agreements by dominant market entities due to their monopolistic or oligopolistic ambitions or position in a market as sellers).

The traditional supply and demand curve does not apply in the way that classical economics recognizes it under these conditions.

In government monopsony, such as the market for items for the Department of Defense and other specialized government agencies, the chances are that only a portion of the available commercial products in the market can be adapted to the requirement.  Very few commercial products need to survive missile impacts, evade guidance and radar systems, or survive through high G escape velocities and then perform in the vacuum of space, to name only some capabilities often desired.  Oftentimes commercial technologies that can be applied must be modified or “hardened” in order survive in the expected environments.

Governmental or public monopsony power cuts both ways.  As it is practiced in the United States under republican institutions, it behaves mostly in a benevolent manner.  The companies that inhabit this space in large part would not exist and achieve profitability if not for the specialized skills and products required.  They are, for all intents and purposes, semi-public institutions.  It is the irony of our times that free market fundamentalists will simultaneously criticize this type of governmental power while acknowledging the need for the end items that are only possible under this arrangement, especially in regard to national defense, but also for any measure of infrastructural, health, and other items required by a functioning nation.

In terms of microeconomics, there are a number of mechanisms that exist to balance out the asymmetrical relationship between the parties, while at the same time certain peculiarities in which the sovereign (the government) maintains its position.  For example, in contracting, the government invites an offer.  The methods of acquisition are requests for quote, requests for proposals, requests for information, etc.  The private or semi-private entity is always the offeror and, therefore, it is assumes freely engages in the activity being contracted.  In addition, only those offerors deemed both responsible and responsive can enter into a contract.  Responsibility includes a host of technical measures but, simply, it means that the firm offering the supplies and services has both the means and ability to perform the work.

The balancing mechanisms that apply in this market are many of the same institutions that provide checks and balances in competition with the Executive Branch: the Courts and the Congress.  The press operating as the Fourth Estate also provides a check in this area.  In addition, economic organizations from industry can petition the government through both direct lobbying and lobbying organizations, and influence the development of policy and standards through membership in professional organizations where certain competencies are shared.

Thus, because of these interrelationships and the nature of the market, competition is oftentimes limited to a few firms possessing expertise that is highly specialized.  Price competition, when it does occur, is oftentimes marginal (or should be) and employs a number of strategies summarized by the phrase “priced to win.”  These strategies can result in market distortions known as “buying-in” to a contractual arrangement.  There may be compelling reasons on the part of the producer/supplier to “buy-in”–denying market share to a competitor, to protect an area of expertise or a skillset or, alternatively, to develop one;  to ensure employment of internal competencies that would otherwise drive up overhead costs due to unutilized labor, apart from the motivation to win business.  For example, on the issue unutilized labor, one can reasonably ask how much it costs to develop an engineer familiar with the airframe of, say, an F-16 and, if lost, how can this expertise ever be reacquired so as to be available when it is needed?  Thus, in government acquisition, it falls upon the Contracting Officer to weigh these issues and prevent outright buying-in, though their expertise and skill in identifying and preventing this type of behavior is certainly not uniform.

Where there is an overlap for supplies ands services across both private and public sectors, there is an opportunity for some competition among firms that provide items based on commercial-off-the-shelf-pricing that responds to a certain extent to the supply and demand pricing mechanism.  For example, in project management, these are usually firms devoted to consulting and to software companies with sharply targeted functionality.  But even here, the expertise required to fulfill some of the needs of the market are unique and, as a result, prices tend to be sticky or inelastic; resisting wholesale competition that would drive a race to the bottom based solely on price.

Furthermore, since acquisition is an Executive Branch responsibility subject to acquisition law, rules, and regulations, their interpretation and implementation is oftentimes influenced not only by operational, but also political considerations.  Since the imposition of sequestration, for example, project managers and contracting officers have been under a great deal of pressure to reduce both costs and prices.  This can often result in government agencies acquiring an inferior or deficient product–or fewer units than needed based on operational requirements–where cost and price pressure is the overriding consideration.  The tenor of looking for cost reduction has been a constant since the early 1980s.  Thus, over the years, a number of “acquisition reform” methods have been employed to counteract the perceived and real inefficiencies in acquiring the latest technology at the most reasonable cost.

Where cost and price pressure is seen to undermine quality, the supply and demand curve in setting price is modified by agencies to require that an assessment of value be determined, and that the item providing the best value, even if not the lowest cost, be selected.  Even in an expansive open market with many competitors there will be a range of similar products offered under range of prices, where the consumer will view several of the products as falling within an acceptable range of value.  This perception of value can be influenced by the asymmetrical relationship between the parties regarding information, or influenced by other psychological factors such as fashion and marketing.  Government monopsony counteracts the asymmetrical relationship of information and psychological factors that tend to distort good market decisions through specialized requirements on offerors–Truth in Negotiations (TINA), representations and certifications of full disclosure, socio-economic and non-discriminatory status, in sensitive areas restrictions on the use of foreign nationals and foreign influence, adherence to U.S. federal labor laws, among other requirements.  For highly complex cost-reimbursable efforts, the offeror must provide cost accounting data and procedures, and ensure that business systems adhere to certain minimum standards prior to award.  Contracting officers can also open discussions and negotiations at will to determine the true cost or value of the supplies and services being acquired–compelling the offeror to provide the necessary information.

Value analysis also acts as a break to downward price pressure.  For example, in my own area of software technology, while it is true that general expertise in areas of programming that 10 years ago were fairly rare–and commanded high salaries–is no longer the case due to policies that promoted globalization, there are some very highly specialized skillsets that cannot be outsourced, at least not yet.  Finding someone who understands not only how to code, but how that code needs to operate in the environment for which it is designed, is a skill that is highly influenced by culture.  The culture can be one of a nation, a people, a market, or a specialty.  In the area of public project management in the United States, it may not be sufficient for someone to come forward with general programming skills without an understanding of the managerial economics that influence the environment.  Instead, they may be called upon to know or be able to learn very quickly the security protocol required for the data being processed, the structure of the organizations in the market, and the manner in which information is consumed and analyzed, as well as issues of cost management, schedule management, risk management, and technical achievement.  Individuals with these skillsets exist but they are few and far between, and command salaries commensurate with their talents and expertise.  From the microeconomic perspective, when there is no longer a demand for this expertise the value will no longer be there to sustain the price established, but there isn’t going to be a lot of price competition while there is demand, so pricing tends to be inelastic.

In public monopsony, the government is the consumer.  As such, this differs from private monopsony such as that recently described by The New Republic regarding Amazon’s impact on the publishing industry, where the dominant market entities are part of the supply chain between producers and consumers.

The management economics of project management under public monopsony, then, leaves no room for pricing adjustments other than what is provided by the contracts that govern the acquisition of the products and services.  Where commercial products can be utilized, pricing is flexible enough in making a difference in terms of project cost reduction for those niches, assuming that they do not require a great deal of specialization or configuration to meet the special needs of the public marketplace.  This creates an environment of both certainty (and hence stability until, at least, instability is introduced by the political system) and a series of constraints on both managerial action and economic flexibility.

As such, there is no room for lack of estimating, scheduling, risk assessments, or performance management as has been pointed out numerous times, but most recently by Glen Alleman in his excellent “Herding Cats” blog.  There is no room for “exploring” because “we don’t know what we need until we build it.”  Recently I attended a meeting with both public and private industry officials discussing how Agile methods can be reconciled with the more structured environment of public project management.  Well, I guess if we forget about budgets and contracts we can all get to that day when developers don’t have to be accountable and we all can pursue imaginary cost avoidance in providing unmeasurable value under the rubric of cults built on catchphrases.  (And a sad day that would be).

In the meantime, for a firm to survive and its products to be sustainable so that it continues to win a share in this market, inelastic pricing that must be established by strong estimating skills in determining what the market entities will accept, an active regulatory regime, and a structured contracting environment–which provide both advantages and constraints–are the overriding factors in managerial economics as they apply to project management under public monopsony.

I will be dealing with private monopsony in a post next week.

*From “Comparative Statics: Changes in the Price of Housing”, section 19.2 from the book Theory and Applications of Economics (v. 1.0).  The chart can also be found here.