Remember when your piano teacher used to insist you practise your scales every single day? Turns out she wasn’t just being a tyrannical tormentor, but a firm believer in the old adage “practice makes perfect.”
It’s a notion that’s played a role in many a success story. Andre Agassi is famously reported to have hit 2,500 tennis balls every day from the age of seven. Michael Jordan spent his off season shooting hundreds of hoops a day in a bid to perfect his jump shot. Even non-athletes need to spend time honing their craft — Winston Churchill had a lisp but became a master orator nonetheless through “practice and perseverance”, while the Beatles spent endless nights performing at clubs and bars in Hamburg before striking it big in America.
With examples abound, there must be some truth in “practice makes perfect” — at least for individuals. But can the same concept apply to businesses as well, especially as firms seek to leverage on advances in today’s technology?
“For the past several decades, companies have been trying to deploy advanced information systems to digitally transform their businesses,” says Jungpil Hahn, an associate professor at the NUS School of Computing who studies organisational learning. Firms do this “to streamline their business processes, reduce error, improve their productivity to increase sales, reach customers better, do analytics better, make better decisions better and so forth,” he says.
In industry speak, this is known as Information Systems Development (ISD). Information systems refers to integrated sets of components — ranging from hardware and software, to procedures and personnel — that collect, store, and process data. Systems provide organisations with the information and knowledge to manage their operations, whether it’s managing their human resources, reaching to potential customers through online promotions, or delivering digital goods.
There have been numerous advances in ISD over the years, with new development tools, programming languages that are easier to learn, and more formalised training in computer science and information systems. But still, ISD is plagued by poor performance, says Hahn.
Only half of all ISD projects are considered successful in delivering the intended systems, he says. The rest often run over budget, fail to be completed on time, or are delivered off-spec. Sometimes, it’s a combination of all three. “The failure rate of a systems development project is extremely high,” says Hahn. “It’s gotten better over the years, but it’s still not good enough…it’s still a challenge to have more persistent, more reliable, more predictable performance.”
Looking beyond the whole
Cue the notion “practice makes perfect”. It’s worked for numerous individuals, so why not businesses too? After all, the idea is grounded in the theory of the learning curve effect: that repeated experiences, or trials, create a collection of knowledge that improves future performance.
“The more similar the tasks are across trials, the stronger the effect will be,” explains Hahn.
But herein lies the problem — there is a “sheer complexity” of ISD projects. Tasks, team members, and the organisational context of clients can vary vastly from project to project. “There are so many different systems and uniqueness in each organisational context that every new project is by definition something new,” says Hahn. “So it’s very difficult to envision how we can benefit from learning through repeated trials, which is fundamentally how people get better at what they do.”
For many years, that was the way traditionalists in the field thought. But Hahn believed otherwise — that “repeated execution of projects” could bring learning benefits to information systems development. So he set about designing a study that would take an ISD project and break it down into its components, analysing those components to see how they influenced learning and success. The approach, he admits, was unorthodox because most studies at the time chose to examine the effects of learning at the project-level, rather than at the components-level.
“What we were looking at is this notion of componentised learning, where even if what you’re doing at the project level is very different, there are still sub-elements within the project that can be reused across projects,” explains Hahn.
Three types of knowledge
Hahn began pursuing this particular line of inquiry years ago while working at Purdue University. Together with fellow colleague Professor Prabuddha De and then-PhD student Keumseok Kang, he launched an empirical study. They obtained access to archival data of nearly 500 ISD projects carried out by more than 2,300 employees of a prominent global IT services company over 20 years. They trawled through the data, looking at three specific project components — domain, technology, and customer knowledge — to analyse how team members’ prior experiences in these areas increased project performance, and to examine what kind of effect the components had on one another.
Domain, technology, and customer knowledge are the three most essential knowledge types for information systems development, says Hahn. Customer knowledge refers to a team’s familiarity with the client, everything from elucidating their preferred working styles and little idiosyncrasies, to understanding and interpreting their demands. Domain denotes the particular area of knowledge or industry within which the client operates. Take for example a project where the client is a financial services company that wants to implement an online wallet, such as DBS looking to set up PayLah!, says Hahn.
“The systems team would need to understand the domain of banking, financial transactions, consumer finance, etc. in order to better understand how people would use such an application,” he explains. “But they would also need to know the technical knowledge of how to develop apps, the mobile environment, and the programming language that are used for online mobile apps — all these things would come into play.”
Learning through components
Analysing the data for these three factors, Hahn’s team determined that “learning does indeed occur at the component level.” Reporting their findings in the journal of Information Systems Research in December 2017, the researchers concluded that prior experiences in the domain and technology components can significantly reduce the cost of a project “in terms of man hours.”
Customer knowledge, on the other hand, helped reduce schedule overruns. “Having a better understanding of your client… whether they’re specific or wishy-washy…allows you to buffer project times to accommodate that risk,” says Hahn. Unfamiliarity results in plans that “are frequently overly optimistic where the team cannot possibly meet the timeline and hence projects are delivered late.”
The researchers also noted that learning effects can be substituted for one another. For example, if team members lack experience with the customer, they can rely more on domain and technology experience to execute the project efficiently. The caveat, however, is that these effects are weaker for more complex ISD projects.
Nonetheless, Hahn says there are important takeaway lessons from the study. Practically speaking, it gives us insights into how best to form a project team to increase project performance. Managers often try to staff a team to match the requirements of the information system, but in many cases that’s unfeasible due to human resource constraints. The study shows, however, that we “don’t necessarily need to have people who are experienced in the same kinds of projects to participate in subsequent projects,” he says.
Theoretically speaking, understanding that learning takes place at the component level “really has tremendous implications for training, and for career planning and progression,” says Hahn. “We could design the training process such that the learning of the different components can be repeated over time… it really challenges our thinking about what’s the best process in terms of learning.”
Today, Hahn continues his research in the field of organisation learning. He’s currently examining the relationship between learning effects and digital entrepreneurship, specifically focusing on online crowdfunding and crowdsourcing.
