So why are we special? - "The Theory"

What is the Culture-in-Action Tool?

Graph on a computer display being pointed to with a pen

The success of a business heavily depends on its culture. Culture influences beliefs, beliefs influence behaviours. Thus, as a large part of many businesses, product development process heavily depends on Culture. For decades, tools and frameworks have been established to evaluate Design, Manufacturing and Quality processes that contribute to developing products. We too use those processes within this tool. Conducted by experienced and qualified engineers, we use TS16949 methodology, cross pollinated with aspects of ISO13485 tools to capture clear and objective evidence, and then comment on the engineering merit behind the evidence. (See FMEA link) However, this is not the only unique benefit. We then use Organisational Culture Analysis methods to triangulate against the objective evidence and provide the social context surrounding the development and manufacture of the product.

With the Culture-in-Action tool, we can provide enhanced product and process auditing and analysis, by using experienced Engineers as auditors, we can also advise if company culture supports your business goals.

Further, through understanding culture within a technical environment, we can also understand behaviours that affect risk assessments in high risk environments, Risk Based Inspection & Reliability Centred Maintenance regimes. We can even use the Culture-in-Action tool to understand stress levels and consequential behaviours in high stress work environments.

How does Culture influence Product Reliability?

Simply put, members of product development teams, like all other functions within a company have pressure to perform. Culture creates an invisible hand that influences how this pressure is applied, perceived and interpreted by the team members. Sometimes the culture encourages positive risk taking, openness, creativity and innovation. Sometimes it does the opposite.

We can see by the multitude of recent real world examples the mess that some company cultures can create. From a theoretical perspective, Wetmore III et al. (2010) discuss the consequences of social interactions and group think on decision making during design review. They cite the Challenger space shuttle disaster as an example whereby an out of tolerance o-ring was allowed to pass through design review due to social pressure. Managers cited the previous launches as evidence the design is sound. This is akin to attempting to test quality into a product rather than design it in, which as has been shown, does not prove reliability. (Jebb, Wynn and Rizvi, 1989). Indeed even after this risk was escalated, many within the project team experienced pressure to conform. Further, social pressure was applied by decision makers for the engineers to prove the design would fail, rather than prove it would succeed. Thus distorting the argument to suit desired outcomes. This is similar to the "narrative fallacy" term coined by Taleb (2008). Indeed Nafday (2009) discusses how engineers may also use narrative, bias, tunnel vision and ignore uncertainty while justifying decisions. It would seem that by reversing the logic within the debate, or indeed controlling the debate (narrative) unknown-unknown risks were made to appear, through narrative, as known-unknown risk.

There was another effect of this pressure to conform. Engineers no longer spoke out. This is similar to the "Mum effect". To explain this effect, Marler et al. (2012) use the back drop of a Korean Air flight that crashes even though the co-pilot and in flight engineer had noticed the pilot’s error. They discuss the “mum effect” and the social pressure to conform to power structures and social norms. Similarly, Smith et al. (2001) and Keil, Mann and Rai (2000) discuss this “mum effect” and its similarities to whistle blowing. While they discuss these phenomena in the context of information systems projects, they identify the studies applicability to other areas. They discuss how engineers, or project leaders, can choose not to speak up about risk to avoid undesirable social pressure, and potentially influence others to stay ‘mum’. Similarly, Keil et al. (2000) conclude that the escalation of software product development projects are heavily influenced by project members behavioural factors. This work is supported by Tan et al. (2003). Also from within a software development project, they go further to identify that these behavioural factors and willingness to deliver bad news to management can be described by organisational culture. However, the work by Tan et al. (2003) suggests that senior managers do not hear about unsuccessful software development projects with sufficient time to react due to either information asymmetry or organisational culture. The choice between the two depending on individual or collective cultures. Even conventional project management theory acknowledges the influence of culture on product reliability. Shore (2008) points out that decision making impacts project success and culture influences decision making. Similarly Belassi, Kondra & Tukei (2007) specifically discuss the influence of culture on New product Development success.

So back to real world examples: This theory sounds similar to the scenario we see played out at VW surrounding the emission scandal reported by Armstrong (2017). He discusses the interplay between engineering decisions and culture. He mentions the pressure on engineers to design emission compliant engines but struggling to do so with existing tools and knowledge. Instead, he describes how they turned their minds to creative solutions, no doubt encouraged to do so, and instead used software to appear to meet standards. Further, and more importantly, his question is less that this could happen, but more that it could be allowed to happen and why did the engineers not fear getting caught by the company or the authorities that control the standards? He makes the observation, that the answer to these questions must be about governance and culture.

The crediting of the situation to culture is re-enforced by Glazer (2016) stating that the VW CEO created a culture of fear, whereby failure is not an opportunity to learn, but something that will not be tolerated. The Boeing Max 737 disasters take a similar tone, as reported by BBC 16 Sept 2020, a congressional investigation concluded that Boeing's "culture of concealment" was a significant factor.

When these examples are considered this way, the theory supports that these recalls sound similar to the Space Shuttle disaster. So too when one uses common sense. The link between organisational culture and product reliability becomes hard to dismiss. Yet, for the most part, companies are still not cognoscente of the benefits of a constructive culture on product reliability when auditing companies. As acknowledged by Kelly-Detweiler (2012) in discussing the link between product reliability and culture, “The only real question to be answered is when, where, and how serious the next Deepwater Horizon, Fukushima, or Columbia will be”

Armstrong, R, 2017, ‘The Volkswagen scandal shows that corporate culture matters', Financial Times, Jan 13.
Belassi, W., Kondra, A. Z., & Tukel, O. I. , 2007, ‘New product development projects: the effects of organizational culture’. Project Management Journal, 38(4), 12–24.
Glazer, R, 2016, ‘The biggest Lesson from Volkswagon: Culture Dictates Behavior’, The Entrepeneur, Jan 8
Jebb, A, Wynn, H & Rizvi, S 1989, 'Robust Engineering Design Post-Taguchi [and Discussion]', Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, vol. 327, no. 1596, pp. 605-616.
Keil, M, Mann, J & Rai, A 2000, 'Why software projects escalate: An empirical analysis and test of four theoretical models', MIS Quarterly, pp. 631-664.
Kelly-Detwieller, P, 2012, ‘BP Deepwater Horizon Arraignments: A Culture That "Forgot to be Afraid"', Forbes, Nov 28
Marler, LE, McKee, DLN, Cox, SS, Simmering, MJ & Allen, DG 2012, 'Don't make me the bad guy: organizational norms, self-monitoring, and the mum effect', Journal of Managerial Issues, vol. 24, no. 1, p. 97
Nafday, AM 2009, 'Strategies for managing the consequences of black swan events', Leadership and Management in Engineering, vol. 9, no. 4, pp. 191-197.
Shore, B, 2008, ‘Systematic biases and culture in project failures’ Project Management Journal, 39(4) 5-16
Smith, HJ, Keil, M & Depledge, G 2001, 'Keeping mum as the project goes under: Toward an explanatory model', Journal of Management Information Systems, vol. 18, no. 2, pp. 189-228.
Taleb, NN 2008, 'The fourth quadrant: a map of the limits of statistics', An Edge original essay, Edge.
Tan, BC, Smith, HJ, Keil, M & Montealegre, R 2003, 'Reporting bad news about software projects: Impact of organizational climate and information asymmetry in an individualistic and a collectivistic culture', Engineering Management, IEEE Transactions on, vol. 50, no. 1, pp. 64-77.
Wetmore III, WR, Summers, JD & Greenstein, JS 2010, 'Experimental study of influence of group familiarity and information sharing on design review effectiveness', Journal of Engineering Design, vol. 21, no. 1, pp. 111-126.

2 men and 2 women 'high fiving' each other in mutual congratulation

What are Recalls?

What types of Recalls are there?

Calipers casually placed on a an engineering drawing

HARD RECALL: Product failures within this category, are post release to the expected end user. This is the traditional use of the term 'recall'. This is likely to be where companies have been, or may be, forced through legal or moral obligations to rectify products being used within their expected lifecycle. We include singular disasters such as the Fukushima Meltdown and the BP Deepwater Horizon within this category, as even though the manufactured quantity was 1, they are still complex products involving complex Design, Manufacture, Quality and Cultural interactions.

SOFT RECALL: These product failures, should could be considered future recalls, or a "near miss". If a significant product defect was found as a product was about to be released onto the market, this would typically be seen as an extension to the release date. Similarly, product that suddenly becomes un-manufacturable. While the reputational impact to an organisation may be smaller than a hard recall, we argue that this should be considered in a similar fashion to a hard recall, as many of the root cause mechanisms are also in play. As such, the Culture-in-Action tool can be used to avoid potential Soft recalls as well as Hard recalls.

What are Examples of Recalls?

What types of Recalls are there?

Two confused tradesmen standing in front of a banner that reads "Plan B"

Examples of RECALLS, whereby CULTURE is reported to have influenced product reliability, function or safety:

VW Emission Recall

R.Armstrong, 2017, “The Volkswagen scandal shows that corporate culture matters”, Financial Times, Jan 13.
M.Sun & J.Hagel, 2020, "Volkswagen Tries to Change Workplace Culture That Fueled Emissions Scandal", Wall Street Journal, Sept 30

Boeing Max 737 Recall

Boucher, 2019, "737 Max Crashes Raised Questions About Boeing’s Culture. Soon Its CEO Will Have to Answer Them", Fortune, Oct 4.
G.Topham, 2020, "‘Designed by clowns’: Boeing messages raise serious questions about 737 Max", TheGaurdian, Jan 9.

Persimmon Homes

R.Neate, 2019, "Persimmon accused of building homes with ‘intolerable’ fire risk", The Gaurdian, Dec 17.
L.De Paoli, 2019, "Persimmon Told to Change Its Culture After Spate of Quality Complaints", Bloomberg, Dec 17

BP DeepWater Horizon

P.Kelly-Detwieller, 2012, "BP Deepwater Horizon Arraignments: A Culture That "Forgot to be Afraid"", Forbes, Nov 28

Fukushima Meltdown

J.McCurry, 2012, "Japanese cultural traits 'at heart of Fukushima disaster'" The Gaurdian, July 5

Spaceshuttle Disasters

2003, "NASA culture key to Columbia shuttle disaster", New Scientist, D.Chandler, Aug 23

Other Useful Articles

Is FMEA based risk assessment valid?

2 men, 2 women at an office desk happily working on a problem with data on a board in the background

Failure Mode Effect Analysis (FMEA), or similar variants such as Failure Mode Effect and Criticality Analysis (FMECA). Sounds scientific right? This is a common risk management tool, often used within an Engineering or Product Development Framework. Described by Sutrisno, Kwon and Hyon (2013, p.54), and enshrined as one of the 5 core Quality tools within the TS16949 standard, it can be summarised as a process that is used to identify systems or components within a product. It then defines how these could fail, (failure mode) what would the effect (severity score), how likely is it to occur (probability score) and how visible, (foreseeability score). Each of these scores is ranked between 1 and 10, 10 being the worse possible outcome. Each score is multiplied to get a total value (Risk Priority Number – RPN) between 1 and 1000, 1000 being a disaster waiting to happen. So the lower the RPN the better. All the risks are then sorted in descending order based on their RPN. A threshold is applied, (often above 200 for some reason?) and risks above this threshold are managed to reduce their RPN. Problem solved right? According to most auditing process, yes!

However, while this sounds very rational, mathematical and quantified, this perception creates the danger within FMEA analysis. Senior managers and directors (and Auditors!) are given the perception that behind this analysis is a group of Engineers, devoid of social influence with thick rimmed glasses that all see Engineering trade-offs, and thus engineering risk, the same way. Truth is truth, facts are facts right? (Tempted as I am, I won’t quote Socrates here.) But, the reality is that engineering is actually a social negotiation. (Lloyd & Busby, 2003) Who gets to decide what the risks are? Who gets to decide the severity, probability and the foreseeability scores? If FMEA is done by a single engineer after the product is designed (or often a QA Engineer in a rush to satisfy an audit), this process becomes make-believe. Acknowledging this, within the TS16949 core tool guidelines advise that FMEA should be completed within a multi-disciplinary team. That is, Design and Production and Compliance and Quality Engineers and the like, before the design is complete. A progressive step that works in a collegial and cooperative organisational culture, but if not, doesn't that just make for an ever bigger social negotiation? Does one person or group dominate the conversation? One only has to look at news articles from the VW emission scandal (Armstrong, 2017) to see how few risks are reported when engineers feel under cultural pressure whereby failure is never an option.

Our premise here is that FMEA and its variants are very useful risk assessment tools that should be used within product development. However, without an understanding of the impact on organisational culture within which the product was developed, their benefit is limited. Worse, it creates the illusion of diligence. Is your auditing team relying on witnessing the existence of and FMEA or other risk assessments to predict product reliability when they can't know the environment within which it was created? If so, be worried.

LLoyd, P & Busby, J 2003, '“Things that went well—No serious injuries or deaths”: Ethical reasoning in a normal engineering design process', Science and Engineering Ethics, vol. 9, no. 4, pp. 503-516.
R.Armstrong, 2017, “The Volkswagen scandal shows that corporate culture matters”, Financial Times, Jan 13.
Sutrisno, A, Kwon, HM & Hyon, J 2013, 'Improvement Strategy Selection in FMEA: Classification, Review and New Opportunity Roadmaps'

Why audit using TS16949 methods?

Is FMEA based risk assessment valid?

Because ISO9000 is not prescriptive on the methods that should be used to evaluate risk, nor how product should be developed, it lacks the rigour to provide an effective auditing framework for arms length assessment of product reliability. ISO9000 only expects that a certified company will have used a method to ensure risks are identified and managed to meet stakeholder requirements, and rectified if they do not. It stays quiet on specific ways to do so. TS16949, (ISO9000 on steroids) prescribes specific Quality tools. For example, PPAP, FMEA, MSA, APQP should be used within a closed loop management system.

We still hold firm to the need for these sort of tools to function within a culture that is conducive to creating reliable product. As such, we feel that TS16949's specific requirement for these tools creates a basis upon which a product recall/reliability audit can be conducted. Similarly ISO13485 and RAMS focus on their own tools and project members competence. "But the company to be audited is only ISO9000 certified" I hear you cry? This is still ok, as we use experienced and educated design engineers with a deep familiarity with these standards, we can still map the companies existing processes to TS16949, ISO13485 or RAMS standards as a comparison. This is also be used to help organisations in their attempt to move beyond ISO9000 to be certified in these other standards.