Measuring Effectiveness in Five Competency Areas
Introduction
The pharmaceutical and biotech stage has changed significantly during the last decade. The growing demand for post-marketing data, the need for customer insights to drive overall strategy, and the more specialized nature of medicine, among other factors, have pushed the Medical Affairs function into the spotlight, moving it from just a supporting clinical role to a leading role in the organization.
In our conversations with multiple Medical Affairs (MA) teams over the past several years, the Learning Solutions capability within inVentiv Health has identified the following five competency areas that are vital across Medical Affairs in general—and specifically for medical science liaisons (MSLs).
Internal Communication
External Communication
Business of Medicine Knowledge
Strategic Thinking and Planning
Scientific Knowledge
This first part of a two-part series showcases why these five competency areas are critical to MSL success and describes ways to measure MSL effectiveness within those areas. Separately, Part 2 will discuss how to address the gaps in MSL effectiveness through knowledge- and skill-based training.
Why Measure Effectiveness First?
Before embarking on any training program, it’s important to identify the gaps and needs to avoid wasting time and money. For example, sometimes a perceived gap doesn’t actually exist, or it exists but doesn’t have a significant impact on the business. Other times, the gap should be addressed by something other than training, such as rethinking the customer engagement model or incentive structure.
The effectiveness conversation should begin with the end in mind. According to Dr. Leslie Miller, an expert on assessment and measurement, “To accurately measure what needs to improve, we need to have criteria—specific outcomes we are looking for.” For example, when it comes to communication, Dr. Miller says, “We might identify the behaviors we would expect individuals to demonstrate when they are communicating effectively.” For each competency area, specific knowledge levels and behaviors need to be defined to understand what “effective” means.
Measuring Effectiveness in Five Competency Areas
Internal Communication
“Communication” is a broad term that encompasses so many aspects of our lives and jobs. For the purposes of MSL effectiveness, we categorize it by the two environments in which MSLs communicate—internally within the company and externally with customers. Measuring effectiveness in these two arenas requires two different lenses.
Internal communication and collaboration with company colleagues within and outside Medical Affairs have been key to the MA organization since its inception. Nearly every major project the MA team takes on—be it launch, sales force training, sharing medical insights, or publishing study results—is in partnership with another team in the company.
Our clients have given us many reasons why MSLs have gaps in internal communication: clashes between personality types and communication styles in Commercial vs. Medical (due in large part to the backgrounds of people who generally take on those roles) and a perceived difference in the ultimate goal of the different teams. In any case, clients seem to agree that internal communication is critical. Katy Shuck, Director, Sales Learning & Development at Janssen Infectious Diseases, offers this explanation: “At Janssen we strive toward ‘Open and Real’ communication and a ‘One Team All In’ approach to ensure candid discussions and collaborations are part of our culture. As the MSLs’ understanding of the various roles within the organization increases, so does their understanding of how their own contributions contribute to the organization’s larger success.”
Additionally, MSLs must be proactive communicators and deliver unique insights to show their value internally. According to Sara Deppe, Senior Manager, Learning & Development at Janssen Infectious Diseases, “Clearly communicating the line of sight [MSLs] provide and how the information they gather is fed through our divisions is critical.”
Measuring gaps in internal communication can be done through interviews and surveys of not only the MA team, but the people across functions they work with. These interview and survey questions should be tied back to defined observable behaviors.
External Communication
Communicating with customers entails not only providing information, but listening to understand needs and gain medical insights on product access and adoption that can help shape company strategy. What’s more, the customer base is changing, and MSLs are no longer just talking to physicians with similar interests and backgrounds to their own. They must now communicate effectively with payer decision makers and hospital CEOs.
External communication is the main responsibility of most MSL teams. This capability can be measured quantitatively or qualitatively. Many companies employ quantitative metrics, such as the number of customer interactions or the number of times a certain presentation or reprint is used. However, companies should ensure that quantitative metrics don’t encourage MSLs to perform activities simply to check the box. Their activities should meet specific customer needs and fulfill a broader strategic goal.
Customer comments can qualitatively measure the effectiveness of MSLs at external communication. The company can survey or interview customers on their interactions with MSLs. Additionally, since the same customer may interact with multiple people from the same pharma/biotech company, internal peers can be a good source of what customers are saying about MSLs. MSLs can also evaluate themselves through behavior-based quizzes that ask questions such as, “How often do I formulate my next point instead of listening to the customer?” If these questions are asked in an anonymous and safe environment, MSLs are more likely to be honest with themselves and the company about their gaps.
Another way to measure MSLs on their external communication is by observing real-life customer interactions or simulated interactions. Dr. Miller provides an example of a simulation: “We might have an MSL experience a very realistic situation where he/she would need to communicate with a disgruntled customer, and an assessor might evaluate the MSL’s communication behaviors during the conversational exchange. Or, we might have an MSL read a hypothetical series of exchanges between an MSL and a customer and, for each exchange, indicate…how he or she might respond next.”
Finally, MSLs can be measured on their insight-gathering effectiveness by measuring the number and quality of insights gathered. Again, for either the quantitative or qualitative metric, it’s critical to define what “acceptable” or “excellent” look like so that there’s a specific standard being measured against.
Business of Medicine Knowledge
As Medical Affairs grows increasingly important within the organization, its team members must understand how the industry works and how outside influences, such as managed care and government, impact the company. MA team members come from various settings that deliver healthcare or conduct research in a way specific to the organization they work for. When they join the pharma and biotech industry, they must learn about the healthcare settings of multiple customers and the connections between those settings.
Knowledge of the business of medicine can be measured in a fairly straightforward and objective manner, using tools like multiple-choice knowledge assessments on specific business topics. These types of assessments measure competence. However, an important part often missed in traditional assessments is confidence—how comfortable someone is with a topic and how well they feel they can explain it to someone else. inVentiv’s MedIQ is one example of a knowledge assessment that addresses both these aspects of knowledge. MSLs answer questions that evaluate their competence, such as, “Which of the following patients is eligible for Medicare?” For each topic, they also answer questions that measure confidence, such as, “Rate your agreement with the following: I have a good understanding of the difference between Medicare and Medicaid.”
One challenge in measuring the capability of business of medicine knowledge is identifying the questions to ask. What do MSLs need to know about the business of medicine to be successful? Most MSLs come from academic or scientific backgrounds where the intricate workings of the healthcare industry were inconsequential to their job. Now that they are key players and important representatives of a pharma or biotech company, they need to know how their industry works, how it is perceived, and what they can do to create a positive impression of their company and overcome barriers their product may have outside of clinical limitations.
For this reason, it’s important that Medical Affairs teams design assessments on business of medicine acumen that address the key topics MSLs need to be fluent in and that they likely have had little exposure to before—whether it’s specialty product distribution, government payers, Medicare, or an overview of common functions of pharma companies. One size does not fit all when it comes to this kind of assessment. Customizing it to the needs of a specific MSL team based on the product or therapy area they support and the related market access environment is critical.
Strategic Thinking and Planning
Knowing the big picture of what the company is trying to achieve and understanding how Medical Affairs departments and individuals can support that goal through their own actions has more importance now than ever before. The MA function, and MSLs in particular, no longer operate in a “scientific silo.” While all members of the core pharma business must think and plan strategically, MA is in a unique position to interact with key customers much earlier than their Commercial counterparts. This early access and collaboration necessitates a strategic outlook focused on learning from customers and sharing those insights with the rest of the organization.
The ability to recognize the company, brand, and functional (i.e., MA organization) strategic goals and develop individual strategic and tactical plans to support them is now a skill widely recognized as important not just for the Commercial side of the business, but for the MA team as well.
Pharma and biotech MSL teams often have gaps not in the ability to strategically think, but in how to execute that thinking in their roles. Sara Deppe from Janssen comments, “Our MSL team is working to become more strategic in their data discussions and the type of information the opinion leader will find valuable. We are in the initial stages of learning how to leverage our data and resources to stand out among our peers.”
Neil Gray, Senior Vice President of Medical Affairs at inVentiv Therapeutics Institute, offers additional insight about contract field medical teams: “We hire for strategic thinking and planning capabilities, but sadly while clients frequently talk about strategy, they buy tactics. Most of our MSLs, while intellectually equipped to plan and think strategically, are focused on the delivery of tactics (the doing)… Having a plan with a sound strategy is the first step in organizing and prioritizing work so that more tactics can be correctly achieved.”
Strategic thinking and planning ability is a soft skill that cannot be completely measured with a multiple-choice quiz. First, it’s important to define what good strategic thinking or strategic behaviors look like. Once these behaviors are defined, companies can look at MSL customer engagement history for quantitative data. They can also seek qualitative input to determine the gap between the ideal behavior and what’s currently happening in the field. According to Dr. Miller, “One of the most common measurement techniques is a 360-degree (or multi-rater) survey. Those who frequently observe an MSL’s behavior would evaluate the extent to which the MSL demonstrates each critical behavior. The MSL would also evaluate him or herself.” For example, are MSLs meeting only with physicians who are product supporters, or are they systematically trying to build relationships with other customers to broaden their scope? Do MSLs change who they visit and what data they present as a result of a new business strategy or market event, or is it always business as usual? Once the best practice and current behaviors are identified, a plan for addressing the gap can be created.
Scientific Knowledge
Whether that knowledge is gained during past work or formal education, or gained while that team member is in role through research or training, the Medical Affairs team remains the scientific brain trust of nearly any biotech or pharmaceutical company. This role has only become more important in the current environment. Importantly, MA team members cannot effectively communicate scientific knowledge internally or externally if they don’t know the content.
Foundational scientific knowledge is fairly straightforward to measure. Simple knowledge-based quizzes and assessments can capture an MSL’s competence in a subject, and a higher-level assessment like a MedIQ can additionally capture confidence.
However, MSLs must be able to do more than just remember and recite scientific facts. An MSL must be able to read clinical trial results and draw the correct conclusions. To do this, they must build on foundational knowledge of the disease state and understand statistical analyses of the trial data and how that influences the conclusions drawn from the study and the design of future studies.
For example, a study might show that a disease is completely cured with a treatment, but if the trial’s population size is very small, or if the treatment results in significant adverse effects, then it would be incorrect for an MSL to tell a customer that the trial was a success without mentioning these issues related to study design and the results. Measuring an MSL’s ability to interpret data correctly can be done by asking MSLs to read an article and report back to their manager or the larger team on the key takeaways and implications. Critical thinking assessments like this can be conducted virtually or live through coaching dialogues or workshops, and they tie back to the key skill of communicating scientific information that all MSLs must have.
Conclusion
On the grand stage of the pharmaceutical and biotech industry, MSLs—the face of the organization to customers—are moving from backstage and into the spotlight. For this reason, measuring their effectiveness in the areas of communication, business and scientific knowledge, and strategic planning is of vital importance. Companies that take a thoughtful approach to measuring MSLs on their knowledge and the ability to apply that knowledge will be better prepared to address gaps and improve performance—ultimately adding value to the company.
Special thanks to our contributing experts
Sara Deppe, Senior Manager, Learning & Development, Janssen Infectious Diseases
Neil Gray, Senior Vice President, Medical Affairs, inVentiv Therapeutics Institute
Leslie Miller, Owner, LanneM TM, LLC
Keith Morris, Managing Director, Practice Area Leader, Medical Practice, inVentiv Health Consulting
Katy Shuck, Director, Sales Learning & Development, Janssen Infectious Diseases