The Importance Of Software Validations For Medical Devices

by Eric Hinrichs, senior principal quality engineer (retired), Ethicon, part of the Johnson & Johnson Medical Device Companies

It’s in the best interest of medical device companies to be as rigorous and thorough as possible when testing software that runs their medical products. If a medical device should have a software issue/fault during a procedure, depending on the type and severity of the fault, the patient’s life could be in danger, or they could suffer irreparable damage. As for the company, put simply, their bottom line, reputation, and future growth may be impacted.

There isn’t any procedure that dictates specifically what should be performed during software validation, as it depends on what the software is supposed to do and what potential interactions could occur. It’s up to the project team to interpret software validation requirements by defining and identifying all program steps, all potential interactions, and the input-to-output expectations. In short, software validation is very much dependent on the team’s experience with and knowledge of the software design, the intended use of the product and software, and the anticipated use and misuse of the product.

I’ve known of software validations that took place where the software experts ran the validation and concentrated on only the prescribed functionality. This approach may guarantee a “successful” validation, but it does not really stress the software functionality as intended or unintended. Software testing should avail itself of all available resources.

Consider using consulting surgeons not involved in the project and have them read the instructions for use (IFU) and then simulate using the device. Observe what they do and don’t do. Observe what instructions were misunderstood or unclear that led to them using the device erroneously. Unfortunately, human nature being what it is, a lot of IFUs don’t get read, which is a good reason to make software as intuitive as possible and not rely strictly upon the IFU. If the device being developed is to be used with products from other manufacturers, or it could be, the project team should test the ways all these devices could or would interact with your product’s software.

These types of pre-validation efforts, along with your comprehensive risk documents, would contribute immensely toward developing an error-free software program. Remember, when filling out your risk documents, don’t dismiss misuses as not being possible. Human error, abuse, and misuse are significant categories when it comes to customer complaints. Don’t hesitate to make use of all the tools at your disposal.

A Hypothetical Scenario

Let’s consider a hypothetical example. Company A manufactures a generator for use with their ablation catheters. They decide to test their generator with two other catheters manufactured by companies Y and Z that are popular with surgeons. There are other less popular catheters on the market (Company X), but Company A decides not to test them due to their low popularity. As to the catheters they will test, they fail to look at ways the catheters themselves could be misused; focusing instead upon how the catheters are expected to be used. Company A decides to not to test these potential misuses because they were considered remote possibilities in their risk documents and, besides, it will save time and money not performing these tests.

In the field, Company A’s software runs well with their catheter and the catheter manufactured by company Y, but not as well with the catheter manufactured by company Z. Company Z’s catheter can develop char if used at too high a temperature; this is not the case with company Y’s catheter or their own catheter. This occurs when the surgeon fails to set the generator’s temperature properly. Since Company A did not stress its testing, it failed to see this difference. If it had, it could have potentially designed this drawback out of their product or warned of the issue in the IFU.

Unfortunately, several hospitals using Company A’s generator with catheters from company X run into software complications during several procedures. The hospitals submit product complaints against Company A, while several patients who suffered complications file lawsuits.

If Company A had tested all potential catheters and interactions, it could have possibly avoided the above scenario. That testing, along with testing all scenarios indicated in their risk documents, may have identified potential failure modes so they could look for ways to correct deficiencies. Maybe it would be possible to program the software to automatically detect which catheter is being used or have the surgeon input the catheter information so the software would be able to set a maximum temperature for a given catheter. Approaches such as these could minimize or avoid reliance upon the surgeon and/or the IFU. This approach may also help minimize potential human error, which, in turn, could reduce the potential for complaints.

Partnering To Find Solutions

While this is a hypothetical example, issues like this can and do occur when medical devices made by different manufacturers are used together during procedures. There are ways to address such potential errors. Some may be considered by management or the project leader as too time-consuming or too costly to pursue, but what is the cost of complaints and lost sales? One problem in advocating for such testing is that it’s difficult to show the benefits/cost savings of avoiding complaints.

One option during product development is to consider partnering with companies that make products your product may be used with. Don’t exclude potential third-party product if you are aware they could be used. It would have been beneficial for Company A to consult with other catheter manufacturers to determine if they have any concerns or limitations with their device being used with their generator and its software. This arrangement would also be a benefit when making software upgrades to ensure all potential devices that could be used continue to interact properly. This is a two-way street. Maybe the catheter companies will be making a design change that could negatively impact their catheter’s interface with Company A’s software. In the end, this type of cooperation may help your medical device be more intuitive to use and, consequently, preferred.

Conclusion

When developing software for a medical device, anything can happen when it is used in the field. Make full use of your risk documents and your company’s medical experts, challenge extreme handling and environmental conditions, and conduct simulations with surgical consultants to bulletproof your medical device’s software. Don’t ignore interfaces if your product is used in conjunction with external devices. Often, this is where most complaints arise. Coordinating with external device manufacturers can eliminate potential risks and errors. It’s your responsibility to detect, identify, and eliminate software glitches and instabilities. The more you test, the more you know and the safer and more efficacious your product will be, setting a standard for reliability and performance. Now, that is a nice place to be.

The Perception of Quality

NOTE: This article was sold to Pathwise.com as a white paper

A friend recommended a restaurant to me saying the food there was to die for. When I dined there I thought the food wasn’t that extraordinary and the service was slow.

Later that month I bought a bookcase that required assembly. Putting it together was a nightmare. Not because I can’t follow directions; I’m pretty handy around the house. No, it was a nightmare because the predrilled holes did not align. I had to make new holes to get the pieces to fit together.

A surgeon tried out a new cauterizing instrument. It was supposed to be far superior to what they were currently using. When the surgeon tried the new instrument. They felt the ergonomics of the instrument were awkward and the cauterizing was not consistent; they went back to their old instrument.

In each of the above scenarios quality was the factor in the person’s experience. But what kind of quality exactly? Was it a measured quality or was it an appearance of quality? We easily categorize quality into two main types.

The first is type of quality is VARIABLE. This type of quality is typically a measurable requirement; where a numerical value can be obtained. An example of a variable quality would be the misaligned holes in the bookcase scenario. The holes were predrilled in the wrong locations. You can usually determine if a quality issue is variable by asking yourself ‘Can I measure the defect?’

The second type of quality is ATTRIBUTE. This type is typically a non-functioning requirement. Examples in the above scenarios would be the ergonomic feel of the cauterizing instrument or the food and service in the restaurant. Attribute quality are difficult to quantify. For example, take the restaurant scenario. My friend thought the food was fabulous, but I found the food to be nothing special. Was it because I ordered something different? Was it because my taste is more or less refined? Take that same scenario with a different person who feels the ambiance is most important when dining out and they come away with a very different view of the restaurant’s quality.

I like to define a third type of quality- perception. This could be argued to be an attribute, but I like to consider it as a separate form of quality because it is associated with feelings. The way the product is marketed and communicated to the user creates an aura or a feeling about a product. In our example of the Acura and Lexus car brands, both brands are marketed as luxury cars- the epitome of luxury. How many people purchase such vehicles because they project success to others? It is this type of intangible that I refer to as the perception of quality.

Where quality is concerned, simply designing a device to accommodate variable quality is not enough. You must look beyond functionality, beyond how durable it is or how well the pieces fit together. You also need to consider attribute quality. How does the device look and feel? Would you buy a car that has paint blemishes? Would a surgeon use a device on a patient that has what appear to be rust spots on its surface? If a device looks ‘dirty’ or has scratches or discoloration upon it, the user may perceive the quality of the device is suspect. For medical devices, an end-user may wonder about the sterility of the device. And lastly, you must look beyond attribute quality to the perception of quality.

Marketing is a good example of the perception of quality. Ads promote the luxuriousness of a product, its amazing properties, or how the product makes one feel. These perceptions drive the experience and drive the perception of quality.

Take as an example product made in Japan after World War II. No one demanded Japanese products. The definition of junk was associated with Japanese products. Japan knew they needed to do something about how their products were viewed. They got serious about quality- all aspects of quality. They drove quality through the introduction of statistical control in manufacturing, they drove costs down through KaiBan and Just-in-Time philosophies. They honed their marketing to create the perception of luxury through brands such as Lexus and Acura. They drove quality in providing what the customer wanted rather than dictate what the customer will get as the Big Three of Detroit was doing. The result was a transformation of Japan from a perception of poor quality to one when they are today the benchmark of quality and performance.

What Japan realized is that quality is not just functionality but also perception. Quality can be both measurable and unmeasurable. We can easily measure defects per thousand and tolerance, but we have a difficult time measuring or quantifying defect avoidance or how many customers we lost because of a misperception or misunderstanding on how a device functions.

This is why quality can be elusive in many aspects. Take the famous Coke dilemma. Coke thought they could improve the taste by altering the formula to make it more desirable over Pepsi. The result was a marketing disaster. Coke reeled from the backlash and had to quickly respond by ‘reintroducing’ the old Coke formula as ‘Classic Coke”. This too was a quality issue. A quality issue related to perception. There was nothing inherently bad about the taste or the packaging that was the issue. What occurred was the perception of the public that Coke had tampered with an iconic product. They wanted their ‘old’ Coke and they were not having anything to do with this ‘new’ Coke.

The takeaway from this is that quality is defined by measured values, attribute values, and the perception of quality. They go hand-in-hand. It is like a three-legged stool. You need all three legs for the stool to function. You need to control the measured values for consistency in performance (variable quality). You need to control the product appearance of quality (attribute) to maintain the look of quality. Lastly, you need to control the perception of quality which is the overall experience the user has with the product. Public perception is key to a successful product. Don’t believe me? Just ask any public figure.

As you can see what we have the most difficulty in measuring – product attribute and the perception of quality- is, in many ways, as important, if not more important, than what can be easily measured- product variation. A company that can successfully control and monitor all three of these aspects of their product quality is a company that can be successful for a very long time.

Eric Hinrichs has more than 31 years in the medical field in R&D on new products and processes, in operations on improving processes, and in quality on leading process and product validations and performing data analysis. He developed and led the implementation of numerous process redesigns while at Ethicon to improve product performance. He authored an ASTM standard on surgical needle penetration, worked on updates to United States Pharmacopeia (USP) medical standards, and helped develop new medical standards for the European Association of Surgical Suture Industry (EASSI) in Europe. He has also co-authored a medical standard for India that is currently under consideration for adoption. He recently released a book on career advice, Perceptions and Expectations.

Effective Project Leadership In The Medical Device Industry

by Eric Hinrichs, senior principal quality engineer (retired), Ethicon, part of the Johnson & Johnson Medical Device Companies

Which of these are true statements about a project leader?

Unfortunately, these statements are all often viewed as the true role of a project leader. I’ve been part of many teams where the above statements were true. I’ve learned there is no magic formula or process that is foolproof in achieving a successful project outcome. Nor can any single training course prepare you for the myriad of circumstances that will manifest during a project.

In this series of articles, I’ll touch upon several areas that I’ve found to be practical advice for project leaders. It may take some time for you to master these, but the effort is well worth it. Hopefully, these lessons and approaches will benefit you in supporting or running your next project.

Team Members Make Or Break The Project

Let’s start with the obvious. A project is only as good as its team. A good project leader will try to get the most talented team members to support the project, though, in many cases, the project leader doesn’t get to pick their team members. Rather, their team members are assigned without any say in the matter. Like the coach of a sports team, the project leader needs to assess the strengths and weaknesses of the assigned team members and figure out the best way to optimize their capabilities. This is key to ensuring they can successfully complete their task(s) with the most autonomy.

I was a project leader where the team members were assigned to the project without my input. For one particular project, I was given a packaging engineer that no one wanted to work with. Their reputation was one of being difficult to work with, their work was typically rife with errors, and they were late delivering their designs, which often required numerous revisions. When they were assigned to my project, I was apprehensive, but what choice did I have? I had to make the best of it. “Good luck,” I was told. I knew I had my work cut out for me with this particular engineer, but I knew I had to make things work. And when the project was completed, the packaging was done early, there were no mistakes or revisions needed, and the packaging engineer wanted to work with me on my next project. What did I do differently from other project leaders to make this particular packaging engineer successful and, as a consequence, this aspect of the project? Here is the approach I used.

At the beginning of the project, I knew I needed to see exactly what the packaging engineer needed to do and what they required to execute each of their tasks. To accomplish that, we sat down and laid out all of the packaging steps for the project Gantt chart. We drilled down so every decision and every step was listed along with all of the inputs feeding into each step along with all of the outputs. As we identified each step, I asked them what their concerns were — what were the typical issues they encountered in completing that particular task? The engineer told me what they needed. I took note, asked a few questions as follow-ups, and ended by telling them I would help them in ensuring their concerns were proactively addressed; they had to make an honest effort to complete each task while I worked behind the scenes to make sure they got what they needed.

After telling the engineer that I was going to do this, imagine my shock when they were surprised at my response on several levels. The first was that I actually asked about any concerns they may have and that I took an interest in ensuring they were addressed. They explained that when they told past project leaders what concerns they had the project leader either dismissed them outright or said they will deal with them if and when they occurred. When the concern did occur, the project leader addressed the issue by putting the engineer on notice that it was their responsibility to get the task done and intimidating them further with the typical “you need to make this date or I will be forced to tell your manager you’re holding up the project.” Seems like the project leader had a different interpretation of support than the engineer. All the project leader’s approach did was increase the engineer’s anxiety, causing them to rush, which only caused them to make more mistakes. That’s not the kind of support anyone would want from their project leader.

The difference between the packaging engineer’s past experience and their experience with my project was that I listened, verified their needs, and took note of their concerns. I was involved because, quite frankly, their success was my success; I had a vested interest in their performance. During the project, I kept in touch with the engineer to ensure they had everything they needed, and I also made sure what they needed was available when the time came. This meant working behind the scenes with other team members and departments to get the information the packaging engineer needed but was in no position to do so.

Manage Your Team Members’ People Skills

Another issue that plays into this is that not everyone on your team has good people skills. They may come off to others as abrupt or indifferent. Traits such as these can negatively impact the support they need. As project leader, you need to recognize your team’s people skills and work to ensure they do not interfere with the project. This is something that is overlooked or ignored too many times by project leaders. When an incident occurs, typical project leaders are quick to blame their team member. Do you think that helps you and the project? If there is a personality conflict, you need to get involved early and work with both parties to ensure the situation does not escalate to the point of impacting the project. When I saw this in my projects, I would participate in the discussions to ensure things went smoothly. Being there as a mediator and as project leader showed both sides that I was aware of what was going on and that it was important enough to me that I was present. It sent the right message to everyone.

Clear Obstacles From Team Members’ Paths

As project leader, you also have the added flexibility and authority to get needed information and resources that team members would not necessarily be able to. Yes, team members would and should try themselves, but sometimes people do not respond the same to someone without authority as they would with someone who does have a level of authority. That is one difference a project leader can make in a project: opening doors that may be closed to team members. Simply put, work to break down barriers to the project’s success.

In essence, as project leader, I made sure the path ahead for the packaging engineer was clear of obstacles and they were working with correct and accurate data. I also made sure I was clear on and understood their expectations by explaining back to them what they told me. This is a two-way street. Conversely, I explained what I needed from them, gave them my instructions and needs, and made sure they understood me as well by summarizing what I requested of them. The result was a smooth, problem-free packaging effort.

What I did wasn’t a magic formula, nor did it take special training or a high-priced training course. All I did was what I would expect from any project leader. As a team member, give me the right tools (data), help remove obstacles in my way before I get to them (get the data needed in time and/or alignment from other team members and support departments so they are ready), and support me along the way (listen to my needs and get me the needed assistance). As a project leader, your role is not to just lead but, more importantly, make the team members’ lives as easy as possible so they can focus on their jobs.

An analogy that may help is how a general contractor (project leader) builds a new home. The general contractor hires the necessary subs (always trying for ones they know are good and reliable – see the similarity here?) and starts the build. The subcontractors will need to see the plans (Gantt chart) to see what is expected and when. Based upon the plans, they will let the general contractor know what materials they will need (data, suppliers, etc.) or what materials they will supply and what concerns (obstacles) they believe might impact their tasks. The general contractor then works to get the materials to the job site and make sure the building is ready for the subcontractor when the time comes. If the general contractor doesn’t communicate properly with the subcontractors, they may arrive at the job site before it's ready for them, there may be confusion as to who was to supply the materials, or they may perform their job incorrectly. The result: the job is delayed, there could be misgivings between the subcontractor and the general contractor (potentially ruining their business relationship, e.g., the team member doesn’t want to work with that project leader in the future), and the project incurs cost overruns and management (future homeowners) is not happy.

A good project leader treats their team members as subcontractors. Team members are given their tasks/roles in the project and they all go off and work on their assignments. The project leader in turn is the glue, or general contractor, that keeps everything moving in the right direction and stays one step ahead of team members.

Conclusion

It really comes down to being an honest, supportive, and involved project leader. This is your team; they are counting on you to help them get things done as much as you are counting on them; you cannot get the project done without them. It sounds trite, but the Golden Rule applies here, too. Treat your team members as you wish to be treated. Simple in concept, hard for some to grasp. Good luck.

Editor’s Note: Check out the other article in this series so far, 3 Common Mistakes Using Gantt Charts For Medical Device Projects.

Effective Planning to Develop Your Medical Device

By Eric Hinrichs, senior principal quality engineer (retired), Ethicon, part of the Johnson & Johnson Medical Device Companies

This article focuses on project execution. The points made will benefit project leaders in avoiding pitfalls that could impact a project’s success. Now that you’re a project leader, your job is to get the project done as soon as possible and get it done right; management is already breathing down your back. Quickly, you assemble your team and develop your plan; now what?

Hurry Up And Go Slow

It’s natural to want to push forward with your project as soon as possible. Action is everything, right? Management sees things happening, the team feels enthused, and all is right with the world. Stop. Action does not mean you are accomplishing anything, especially in the beginning of a project. Often, in a project, it is the decisions made in the beginning that could mean success or failure for your project. What makes things more difficult is that planning is most critical early in the project, when you know the least about what you need to do or what issues you may be confronted with.

Let’s look at this further.

You have a project that will need to be run in several plants. When you get to the point in the project where you need to run tests in one of the plants, you discover that they do not have adequately trained personnel to perform the testing, or you find their testing equipment does not have the range to test your product. Suddenly your project is on hold, losing valuable time when you can least afford it. Management was expecting this phase to be done – you told them in the last project update, remember?

This is where that rush in the beginning of a project sometimes backfires. I like to say, “hurry up and go slow.” This is meant to convey that you should move a project along as quickly as you can but not at the sacrifice of thoroughly planning out the stages you need to execute; instead, make quick but well thought out decisions.

A well-planned project will go through various scenarios in the planning stages and pull in team members to find out what will be needed for each phase of the project. Even more valuable is to pull from each team member’s experience to see what issues other teams they were on ran into that you can guard against. This same approach is useful to execute with the plants where you will be testing. Check that they have the trained personnel, the proper equipment, tooling, and capacity you will need. Each of these approaches helps your team avoid delays that are avoidable through proper planning. Having a well detailed Gantt chart and thoroughly vetted resource needs will save your project from unexpected delays.

I’ve experienced teams that jump right into projects and assume resources will be there when they are needed, only to find that was not the case. It is for this reason teams need to carefully plan up front and be sure they have thoroughly planned out what needs to be done and what needs to be available so they can move smoothly through testing/validation, etc., to a successful product launch.

Forewarned Is Forearmed

Planning also means getting out in front of the project’s schedule. This means letting whoever is needed for upcoming tasks know that you are almost ready for them. I built my own home and hired and directed my own subcontractors. Things went smoothly during construction. The main reason was I would meet with subcontractors weeks before they were needed and review with them where things were. We would also go over what needed to be done by them and what, if anything, they needed to get their job completed. This approach worked very well in ensuring the job site was ready for them. It was also useful that I gave them advance notice so my job could be slotted into their schedule. Nothing is worse than when you need a subcontractor or a plant and they tell you they have other commitments and cannot make your date because they weren’t aware you needed them for those dates or times.

To avoid these situations, inform everyone you need to work with well ahead of time and review your needs and their needs to gain alignment.

Verify, Don’t Assume

The worst thing you can do as a project leader is assume you have the materials and resources available when they are needed. Proper planning involves verifying material and resources are obtained or reserved for when they are to be used or employed. I’ve been in a situation where we had purchased material for the project, only to have the plant use the project’s material for their own needs because they had run short. We did not find this out until we went to start validations in the plant. They did not tell us, nor did we check to make sure we had everything for the planned testing. It was a hard lesson for the project team to learn and it was an avoidable delay. You can imagine how management took the news that our project was delayed until we could get additional material.

Conclusion

Plan thoroughly in the beginning of a project, anticipate potential problems that are likely to occur, inform your resources prior to needing them, and gain their alignment as to timing and resource commitment. Verify materials and resources are available before they are needed. If you focus on these aspects of a project, the likelihood that you will have a smoothly running project will be high.