Test: In-House or Outsource?
Today, every supply chain executive understands that core competencies define the success of an endeavor. Having the right people doing the right tasks will result in advancement and profitability. Most enterprises long ago recognized, for example, that business-critical technologies like enterprise resource planning (ERP) solutions can be developed, implemented and, in many cases, managed better by dedicated providers than by their in‑house IT organizations. Additionally, some technologies are, of course, so specific to the enterprise and its needs that they require, at minimum, extensive involvement by internal specialists.
Deciding which technologies fall into which category is a very demanding and risk-intensive process, and test engineering implementations is no exception. Decision-makers can simplify and rationalize the decision process by applying basic supply chain principles – that is, by defining the core competencies that are needed and determining who possesses those competencies.
The Impact of Test Engineering
Manufacturers worldwide have come to recognize the critical importance of manufacturing testing. State‑of-the-art innovations are happening every day, in every industry. These creations are representing the company's brand and informing a customer if they should be trusted. It is of utmost importance that the quality that is being sent out into the world matches the customers' expectations. Meanwhile, these same products need to be affordable enough to make a profit yet priced in an affordable way.
Designing the right test solution (and implementing it well) is the driver for both quality and cost. One must consider the knowledge and experience that is required to do it properly. On any given day a test project may require hardware, software, optical, RF, robotics, and automation expertise. To assure success it is important to take a step back and consider who is in charge of what. While a company may employ the smartest medical experts or automotive designers, are they the best team to design a vision inspection system to deliver high speed and flawless results? A non-specialist approach tends to slow down production quickly by sticking with manual verifications and subjective results. Reliability and repeatable are the main ingredients for better test. A dependable system delivers both.
The Repercussions of a Quick Fix
One high-profile example illustrates the severe importance of test in manufacturing. The rollout of a new smartphone was compromised by design flaws and supply-chain issues. According to the New York Times (Jan. 2017): “Samsung and outside experts said batteries made by two suppliers contained flaws that allowed the phones to overheat and in several cases catch fire. But they also cited what they said were flaws in the design of the phone, including an unusually thin lining between the electrodes of the battery.” These issues weren’t discovered until the devices were in the marketplace and Samsung took a multi-billion dollar hit. Thousands of cellphones had to be returned to the manufacturer and the impact on the profitability of the newly released product was immediate. Neither the batteries nor the device itself had undergone the proper method of testing, and clearly the communication of the test data was flawed.
To avoid such scenarios, the following must be considered:
Total Cost of Ownership (TCO)
Enterprises of all types continue to operate under many financial and resource constraints. Today’s economy is still recovering from the worst economic downturn since the 1930s along with the relief methods provided in aid of the COVID-19 pandemic. Businesses remain cautious about spending on new initiatives and every budget decision comes under scrutiny. Any major deployment involves a significant financial investment, and manufacturing test analytics is no exception. Whether developed in‑house or outsourced to a dedicated “pure-play” provider, a truly capable test solution may cost hundreds of thousands dollars when deployed across an entire enterprise. For this reason, it may be tempting to try to develop capabilities in-house, particularly if some basic functionality is already in place.
It’s important to remember that the TCO of such a solution – which in some cases may seem to offer initial cost savings – can increase sharply over time, especially as the supply chain and its enterprise-specific requirements grow and change. Under conditions like these, outsourcing may ultimately deliver much lower TCO than internal development. Enterprises facing this decision also need to consider the type of investment involved and how it will be treated by the business. Internal development is likely to be considered a capital expenditure for an extended period, while an outsourced solution will, over time, come to be treated as an operating cost. Moreover, an external provider may offer different development, delivery and pricing models to its clients. The most important factor, however, is probably the scalability of development. An external provider, unlike an in‑house organization, can spread development costs across many locations and many clients, and pass on the resulting cost savings, which may be considerable. Manufacturing decision-makers clearly recognize the potential for cost savings that can be realized by outsourcing key functions. In fact, a recent survey from Deloitte highlighted that companies are outsourcing more as a result from COVID-19. The number one motivation for this move has been cost reduction.
Understanding the Costs
By breaking down the TCO it becomes easier to determine where costs can be cut and decide what kind of investment needs to be made to get the highest ROI. In the case of test, cost needs to be separated into 3 categories: Non-Recurring Engineering (NRE), Recurring and Operational. NRE costs will be known in advance and will only happen once. NRE will include all the development time required to design a new test station. This also factors in the time to design the right strategy, get the tools that will be re-used for repeats, training, etc. Recurring costs will also be known in advance and will need to be spent on the required material, labor & logistics for each station. Lastly, operational costs are the amount to pay once the machine is up and running. This amount is generally affected by the amount of labor required to run the station, the amount of floor space it takes up, the power consumption, etc.
Numbers Don't Lie
For example, a manual test station that can test 5,000 UUTs per year and requires a lot of operator assistance. It will certainly have a lower NRE and recurring cost than a fully automated station that will test 20 times that amount. The manual station will cost a lot more to operate and produce a lot less. If that satisfies the enterprise output requirements, that is a viable solution. Consider the following scenarios:
→ Company Requirement = 100,000 UUTs/year:
→ Company Requirement = 20,000 UUTs/year:
Note: Estimations exclude cost of floor space, power user and operator training.
The availability of the development resources required to undertake major new initiatives remains a major concern for many enterprises. During the worldwide economic downturn, most organizations faced steep budget and staffing cuts, and even in the current recovery phase, they’re still routinely expected to do more with less. Under these conditions, which seem unlikely to improve dramatically in the foreseeable future, enterprises considering new endeavors must ask themselves two crucial questions:
- Do we have the resources we need to implement and manage the new test project?
- Do we have the operational setup to support the technology?
The first question is, in some respects, easier to answer. Highly specialized technologies like manufacturing test require equally specialized skill sets. The organization can determine whether it has those skills in place or can acquire them quickly and cost-effectively enough to justify an in-house implementation. But as stated above, test engineering requires a myriad of proficiencies. It is likely that an organization will hire based on the requirements of their product, not necessarily the means to test it. The people may be there, but are they the right people to perform the task well? Training and ramp-up is always a possibility but the downstream “ripple effects” of a large-scale initiative are far more difficult to calculate. Every professional who is committed to learning the new technology must be pulled off another project. The impact on the organization’s overall performance is likely to be far-reaching.
The challenges of a manufacturing test solution implementation extend far beyond the product's technology. The first step in a successful test project is a complete test specification plan. By laying out industry requirements and product expectations, a full test scope can be defined. The main challenge is that the test requirements will rarely match the application's purpose. For a product to work in the real world, outside considerations must be designed into the test plan. Questions that need to be asked include, what is the environmental impact? How many times can a button be pushed before it breaks? If a message is sent over Bluetooth, is it received on the other end? How will the casing of the product affect its performance?
And these considerations must accommodate extreme scenarios. As an example, electronics designed for use in a car must withstand severe temperatures & climates. A company may be capable of designing the best amplifier on the market, but do they know how to create a climate chamber to determine if the amps can stand temperature changes from -40° C to +40° C?
Additionally, testers must be designed to be efficient & cost effective. Often they can accommodate multiple units simultaneously, or different product variations. To identify where test instrumentation can be re-used or understand how to inspect 10,000 units within a minute requires expertise & exposure that may not be standard in-house knowledge. These benefits are rooted in automation and is a skillset of its own.
The ability to rapidly and efficiently adjust to changing conditions is a critical component of any organization’s value proposition. This makes scalability is a key selection criterion for any manufacturing test solution. This technology, and the human and organizational resources needed to support it and use it effectively, must be able to “ramp up” in response to sudden sharp increases in demand and to shift operations to different locations when operating conditions change. And it must be able to accommodate the often-immense amounts of test and quality data that need to be aggregated.
The largest, most experienced manufacturers focus intensely on ensuring the scalability of all their basic operations. But even these highly capable enterprises may be challenged to adequately scale technologies as specialized as test. Companies that are already struggling with these issues in other areas – and the repeated recent supply chain failures show that many are – may decide that adding another layer of uncertainty about scalability represents an unacceptable level of risk. Manufacturing is composed of countless moving parts. It is the one place where redundancy is not necessarily negative. Whatever can be reused and repeated offers simplicity, adaptability, and capability.
Common Core Test Stations
A common core or generic test station is a flexible solution that accommodates multiple products with easy ramp up or ramp down based on demand. By analyzing product requirements, these stations are design to group all of the repeating test functions into the base of the station. The right generic station is a bigger investment to make upfront. The common tests often require expensive instrumentation for execution. Then, less expensive fixtures are designed to be product specific. They can be swapped in and out to adapt to the product type. Investing in the proper design in the core of a test station delivers major reuse of expensive equipment and carries across a corporation from design to repair.
A good core will include all the expensive instrumentation plus the initial NRE. From then on, any future investment will be made in the much less expensive fixtures. Again, these fixtures accommodate all the unique requirements of each product type. This allows you to reuse the expensive core repeatedly and potentially saves hundreds of thousands down the road.
Is a Common Test Platform the Right Path?
Not every test setup is a fit a generic test station. It is important to understand the manufacturing requirements in advance, along with a solid grasp of the technology. Overlap is required in both to make it worthwhile. For example:
One manufacturer has 2 products, A & B. Product A requires a throughput of 140% and Product B requires 120%. As a result, each product would require 2 stand-alone stations as buying 20% of a system is not an option. By investing in dedicated test stations, they would have to spend $250,000 per system plus 4 fixtures at $15K each. This comes out to a well over million-dollar investment plus 4 units of floor space. Instead, what chose to do is invest more in a generic core at $300K for 3 stations, plus 2 fixtures for product A and 2 fixtures for Product B, priced at 20K per fixture. As a result, they will spend $980,000, saving 7.5% of the initial purchase price and only occupy 3 units of floor space. Additionally, every follow up purchase down the road is set at 20K.
|Dedicated Station||Common Core Platform|
|4 Stations||1 System|
|$ 1,060,000||$ 980,000|
|Per Station||Per Station|
|$ 265,000||$ 20,000|
A manufacturer has a major shift in the demand for their product. One of their products is being phased out while another had grown by 100%. Luckily, they had originally bought a common core test platform. Now, instead of reinvesting in a dedicated test station to accommodate the increase, they can simply purchase an additional fixture for the growing product and leverage the station from the discontinued one. The fixture design will be readily available with their test partner, making this a fast and easy way to accommodate a major change. This saves the manufacturer over 10 times what they would have spent on a stand-alone station and adds no machinery to the floor.
|Dedicated Station||Common Core Platform|
|1 Station||1 Fixture|
|~$ 150,000||~$ 15,000|
|Per Station||Per Fixture|
|~$ 150,000||~$ 15,000|
A repair center is the perfect candidate for a UTS. Typically, one test station is required for each product type. Depending on the manufacturing variety which will only grow over time, this is substantial. This equipment needs to be usable for long periods of time to be able to properly service the clientele but is only used less than 10% of the time. By investing in several test cores, future investments will be limited to less expensive fixtures. Over a 20-year period and multiple product iterations, it only makes sense to make the equipment as reusable as possible.
|Dedicated Station||Common Core Platform|
|1 Station||1 System|
|~$ 125,000||~$ 200,000|
|Per Station||Per Fixture|
|~$ 125,000||~$ 25,000|
It is always a worthwhile exercise to conduct a feasibility study for future test endeavors and determine is a generic station is an option.
Many manufacturers must meet rigorous standards and specifications that are specific to their industry verticals. Aerospace and defense contractors, for example, have very detailed import/export restrictions and intellectual property protection mandates. Pharmaceutical companies must deal with exacting regulatory compliance standards, especially in product development reporting. Nearly every original equipment manufacturer (OEM) has service-level agreements (SLAs) with its customers and partners that are unique to its industry or its particular business. This degree of specialization leads some manufacturers to believe that only they can design, implement and manage a test solution that meets their needs.
In some cases, they may be right. But pure-play providers have usually developed their solutions for specific use cases and deployment scenarios, and many have considerable depth of expertise across many industry verticals. Moreover, some have highly experienced consulting and services businesses that give them the capability to adapt their technologies to new use cases.
Experienced test partners gain a huge knowledge base when dealing with multiple industries. It keeps them ahead of the curve. Certain industries like medical devices are slower to introduce new technologies into manufacturing due to the regulatory compliance standards mentioned above. When dealing with a company that has extensive experience in both life sciences & less regulated consumer devices, they will already be familiar with the necessary compliances and the newer technology that is being implemented.
Modern manufacturing operates across so many locations, regions and time zones that it requires something very close to around-the‑world, around the‑clock support. Most enterprises today are already stressed by the demands of their extended operations and this may make outsourcing functions like manufacturing test attractive. Of course, the outsourcing provider must be able to offer the level of support the manufacturer requires, in the appropriate regions, time zones and languages. A worldwide presence – closely aligned with the manufacturer’s own needs – should be a key selection criterion for any test solutions provider. The cost of a factory shutdown is never worth the savings a lesser test solution may provide.
Product testing has become increasingly complex due to the convergence of many different technologies into single products like smartphones, wearable devices, automotive infotainment systems, and medical devices. In addition to the obvious challenge of testing a greater amount of technology in increasingly smaller devices, there is also the fact that product competition is strong while shelf life is shorter and shorter. Many of today’s electronics manufacturers are therefore optimizing their test architecture and test routines to cover more product functionality as quickly as possible. By using standardized test software such as LabVIEW and NI TestStand, they can implement batch and parallel testing to speed throughput, optimize instrument use, automate standard operating procedures (SOPs), and easily gather test result data to further improve processes. It’s clear that today’s complex in-vehicle systems require new test approaches.
An important component in thorough product testing is automated test equipment (ATE) including assembly test stations, functional test stations, and environmental stress screening (ESS) test stations. They are used to verify assembled components such as semiconductor chips and boards, or to functionally test partially or fully assembled units. Additionally, depending on how and where the product is used, there may be specialized test stations, fixtures, and chambers used for testing small unit samples to ensure that they operate when vibrated, dropped, immersed, or exposed to electromagnetic or extreme temperatures, and so on. Units that fail at any stage are typically routed to repair stations for troubleshooting and reconditioning.
ATE's provide reliable and repeatable results that cannot be replicated through manual testing in the same time frame. Automation ensures every test procedure, sequence and step is followed to a tee for every device. It is physically impossible for two human operators, no matter how well trained, to provide the same quality of results. Performing the same task over and over is redundant and cannot be done the same way 100% of the time. Automation provides repeatability, reduces variability and can achieve both precision and speed that simply can’t be done by a human alone. Automated vision inspection, for example, can detect the tiniest flaw in microseconds. For a person to perform the same task, a lot of time is wasted focusing and re-focusing a microscope. Automation easily sorts out defective products, and therefore also simplifies meeting compliance requirements in more stringent industries.
The Bottom Line
None of these decision factors can be considered in isolation, and different enterprises will weight them differently, depending on their specific requirements. For some highly specialized product developers or manufacturers, the need for deep domain knowledge may outweigh economies of scale as a technology selection criterion, and suggest that an in-house solution is the safer path. For others, concerns about project complexity and scalability will point in the direction of an outsourced solution from a dedicated provider. And for every enterprise, whatever delivery model it chooses, cost – both initial and TCO – will certainly be an important consideration. Ultimately, how the selection decision is made matters almost as much as what decision is made. Key decision-makers from across the enterprise, and sometimes from outside the enterprise, must be involved, and their interests and concerns must be taken into account. Essential requirements must be identified, along with the core competencies that can address those requirements. Everyone involved must make an honest, clear-eyed assessment of what they need from a test solution and how it will impact their roles and responsibilities. If they approach the selection process in this way – the way they approach everything else involving the global supply chain – the right decision will be made.