Select the Right Partner for your Next Employee Survey – 20 Considerations!

Summary: Here I provide key topics and questions that HR professionals should ask potential survey providers before embarking on large scale organisational development projects. From company background, data capture and project management to analysis techniques and post-survey action planning support, these questions will help you become a more informed buyer and set you on your way towards a more successful employee experience improvement project.

“It is not the answer that enlightens, but the question”

An indispensable skill in any buying process is the ability to pose the right questions to vendors. There are no right answers to wrong vendor selection questions, so we need to make sure we get these right the first time round. Successful HR professionals ask better questions during HR vendor selection processes and, as a result, they get better quality answers and so are better able to make the right long-term partnering decisions.

I have been conducting employee engagement surveys for the past 15 years, so I am frequently approached by HR colleagues seeking advice on how to select a good survey partner for their coming employee engagement project. How to select a good employee survey provider is a fair question.

The relationship between an organisation and their employee survey vendor is like a marriage. Once your organisation implements an employee survey with a specific provider, you want to keep that relationship going for the long term, not only because of the historical data and trends between surveys, but more importantly because of all the time and effort invested in that specific vendor relationship - the mutual knowledge transfer, good working synergies and joint approaches to solving problems that have been gained during the time doing business together.

So, when I hear that an organisation wants to change survey provider alarm bells start to ring: something must have gone awfully wrong! – unless of course, you have simply been working with that same provider for too long, without much innovation in say the last three years (!?), and feel that it is time for some fresh ideas, new thinking and approaches on the employee research and post-survey action planning front.

I normally start my vendor selection conversations by asking: ‘Why are you planning to change your current employee survey provider?’ Through these exchanges I have managed to distinguish and distil some precise questions which are key to identifying specific client requirements, and more importantly, the necessary service arrangements expected from employee survey providers.

There are a large variety of questions that you can ask vendors, but unfortunately, both you and the vendor will not have the time to deal with them all, so it makes sense to be focused and think beforehand where and how to start your vendor investigation process. Here are my top tips for concentrating on the right themes and questions in your next RFP and subsequent shortlist of vendor selection interviews.

1.      Expertise with people and not just data: Survey vendor background and experience is important to establish not only how long a company has been providing surveys, but also the depth of experience its experts bring to the table. This may sound obvious, but make sure the vendor that you are choosing is not only skilled in all methods linked to employee data collection, but also understands what the data is for and how it is to be used. A professional employee research vendor should be able to provide examples of a wide range of projects representing an array of markets and survey sizes. Key questions involve: How long has your company been in business? How long has your company been providing employee survey research services? What are considered the core services provided by your organisation? What proportion of your company’s total annual work is focused on enterprise-wide employee survey research? What is your organisation’s survey research experience in major regions around the world? What percentage of your staff work purely in employee survey research? Beware of large consultancy firms where employee surveys are just an inconsequential part of their overall business. There’s no point in being at the forefront of technological innovation if the vendor has no clue how to convert real insight into measurable business change. It’s not about fancy online graphics and colourful survey reporting platforms. Make sure your prospective vendor is an expert in the psychology of people and the analysis of employee data. This line is especially relevant today with so many market/consumer research organisations entering the HR arena claiming to be experts in employee research. 

2.      Current clients and references: Confirm your vendor has all the necessary credentials in ‘actionable’ employee research. Some vendors will show you an impressive list of clients and big brands. Sometimes they claim to have worked for big company names, but they may have only done a very minor piece of work for them, possibly unrelated to employee research, or even worse, they may have never actually worked with those big brands at all. Always seek assurances about the quality of work – ask for serious case studies, references and testimonials of existing clients in your sector dealing with projects that are like yours. Ask them to provide at least three references for employee research projects of a similar size and scope that they have managed in the past 18 months. Ask to talk to the project managers of those initiatives.  

3.      Project management support: This is the difference between success and failure. When your team works with an outside provider, it is essential to understand the vendor’s overall approach, plans for communicating with the team and the capacity and willingness to train your staff. Make sure you ask how they plan to manage and support your different survey teams around the world, what training they are planning to provide, how they plan to manage and co-ordinate project-related information and timelines among your different business operations. Also ask what sort of assistance they can provide with pre- and post-survey communications – request examples. Make sure the vendor will support you throughout the project and will not abandon you to deal with the whole initiative on your own devices.  

4.      Geographic location of the vendor’s survey and customer support teams: When it comes to implementing and acting on the survey results, it is important for vendors to be able to offer local support in different parts of the world. Some large consultancies claim to have offices all around the world. Yes they do, but sometimes none of these specialise in employee research at all. Don’t be impressed by consultancy firms claiming solidity by their large numbers of employees and offices around the world. The reality is their employee research team might be very small, just a tiny proportion of their overall workforce, and may only operate in a very few locations around the world. Also, some vendors may have all their operations abroad in developing countries where they pay cheaper labour to do the work for them. Many times, clients end up dealing with people on the other side of the world that at times have very poor English skills and incompatible business culture traits. Time differences with these locations is another challenge – make sure all your different teams will have access to your vendor’s customer support during your local business working hours. Shipping all your employee data to these developing countries is also a concern due to different data security standards. Problems with quality can also arise if this offshore provider doesn't have proper processes and/or is inexperienced in working in this kind of outsourcing relationship model with clients they have never met. So, ensure all those offices around the world specialise in employee research and operate in the countries where your business units are located. 

5.      Ask about the background of all team members to be allocated to your project: Some vendors send their top sales team with the sole purpose to close deals. These individuals are very persuasive sales people and impressive speakers. They will present themselves as experts who will support your project and will be there when needed. The reality is that once the project is sold, you will not see their faces again and often you will need to deal with very junior and inexperienced staff. In many instances their sales representatives are external contractors who are paid a commission for the deals that they win. These people are not involved in project delivery and/or account management and, on many occasions, may over-promise delivery to win the deal. Make sure you know who you will work with, ask for the CVs of all the people to be involved in the team and their respective roles. Determine the structure of the team that is to support you and the roles and responsibilities of its members. Identify who will be your main day-to-day point of contact and make sure you feel comfortable working with this person. Does this person have sufficient project management experience or knowledge to offer advice on questionnaire content, pre-survey communications and the implementation of your post survey activities? Make sure you ask them to provide precise details on all expected team members’ expertise and expected contribution to the project. 

6.      Confirm your chosen vendor does not outsource all their technology: It is not ideal that your chosen vendor relies on third parties for all their data collection and reporting technologies. If this is the case, their capacity to fix your problems or respond to your requests will be much slower. Since the technology outsourcing provider will also work with other various customers, they might not give full attention to your company. This may result in delays and inaccuracies in your employee survey work output. Apart from data security concerns, also think of risks such as bankruptcy and financial loss of the outsourced organisation; these cannot be controlled by your vendor. 

7.      Certify all services and technology have been properly tried and tested: There are vendors that will tell you that everything can be done for you. Yes, everything can be done, they can tailor or create new technology for you, but of course, will use you as a guinea pig not only to develop untested products and services, but also to fund them. They will try out technologies that have never been tested before - that is very risky. Note that the only parties who will benefit are the vendor and their future customers. If you are trying new products with your vendor, your team will have to leverage a diversified test strategy including automated, manual, and exploratory tests to build quality at all levels in the new software being created for you. To be sure you get full view of how the technology works, request demonstrations, webinars and meetings in the RFP process. As part of the deal, you may also want to consider piloting the new technology with a small sample of your employees.  

8.      Sampling of survey participants: If you wish to implement various pulse surveys throughout the year and avoid survey fatigue among your employees, your chosen survey vendor must be skilled in the evaluation and implementation of robust sampling processes to ensure adequately representative employee populations. If all employees are not participating in a census survey, you need to ask your potential vendor how they plan to define the sampling strategy to ensure a random selection and representativeness of the survey results. If sampling is to be used in some locations and census in others, you need to ask how your vendor is to weight your data prior to tabulation to restore proportionality. 

9.      Robustness of external benchmarks: Many times I have HR colleagues telling me about the wonderful employee normative databases being offered to them- hundreds of companies, big brands in specific industrial sectors with thousands of respondents. When you face these claims, you need to pause and ask: How often do they update their benchmarks? How old is their data? Who are these respondents? How was the data obtained? Is it panel research data? Is it the vendor’s client data? Is it representative of the market that you want to evaluate? Is this data dominated by multinational organisations or is the benchmark a true representation of the industry? Is this benchmark dominated by a few players in the industry? Does the vendor weight the benchmark data? Answers to these questions will give you an idea of the validity of those benchmarks that they are trying to sell to you.  

10.  Survey strategy: Before the vendor organises your survey, they need to discuss with you clear research objectives: What is the aim of the work? What is the overall purpose of the study? What are the desired outcomes of the project? What are the general intentions of the research? These should be clearly and concisely defined by the vendor, otherwise, it will be very difficult to act at the end of the survey. Lack of research objectives will offer an amateurish picture of your employee research project. This lack of people insight strategy is embodied by less desirable vendors; they propose quick implementation to cash in with the project as quickly as possible. Instead a serious provider would challenge your decision to launch a survey until the survey objectives are well defined and the right survey questions and the right implementation plan are in place. In this respect, it is a good idea to ask prospective vendors how they approach their actual survey strategy. 

11.  Strategic alignment of survey content: The survey questions may have no relation to the needs of the organisation - these may not be properly aligned with your organisation’s business strategy/direction and industry at all. A quality survey will address your company’s goals and unique cultural considerations while allowing flexibility and adaptability in the data gathering process. In this respect you may want to ask your potential vendor to describe their firm’s approach to developing a survey focused on your company’s strategic priorities. Do they only work with ‘off-the-shelf’ surveys? Are they able to offer a standard survey to be used by all areas of your company with the option of customising additional questions for different divisions, functions, units, etc.? For each international site, how would the vendor review the questionnaire for inappropriate and/or non-applicable terms? How would they work with you to modify the questionnaire accordingly? How would the vendor have the survey contents and reporting platforms translated? What standard response scales does your vendor use? How would your vendor approach the incorporation and qualitative analysis of open-ended (comment) questions? In terms of data segmentation, especially when it comes to matrix organisations, vendors need to be experienced in linking your current organisational hierarchy to your exact reporting needs. 

12.  Research model and links to performance metrics: In addition to providing questions that are relevant to your organisation’s strategic priorities, an experienced vendor survey approach should be based on a sound, validated research model that considers cultural variation and directly links to your very specific business performance metrics. You may want to ask your potential vendor: How does their organisation define and measure Organisational Trust, Organisational Effectiveness, Employee Engagement/Employee Experience? What is the rationale for their approach? How do they consider cultural variation when assessing employee experience? How does the vendor identify drivers of Employee Engagement, Retention, Efficiency and Productiveness? What is the vendor’s approach to linking measures of Employee Experience with your own key performance indicators/human capital metrics? 

13.  Tried and tested survey items: Some vendors might be inexperienced in HR matters and may not have a clue about employee survey design. They may deliver questionnaires that are complicated and confusing, and even useless for acting on the survey results. The survey could be full of double-barrelled items which measure two different things at the same time. They may use both negatively worded and positively worded statements which are intermingled in the survey. They may use different rating scales. These are just examples of poor survey design that leads to confusion and frustration on the part of the survey taker. It also may provide mixed results, as participants in different geographical locations may interpret the questions differently. Most importantly all items need to be actionable, you will need to be able to do something with your employee feedback. To ensure success, the final questionnaire needs to be understandable and straightforward for employees of all organisational levels, short ‘tried and tested’ items which are simple and easy to read. 

14.  Transition from previous survey vendors: If switching vendors, you may not want to lose all the good work that you have done in the previous years. You need to ask your candidate vendor how they approach integrating previous survey work to ensure the comparability of your historical data. What experience do they have with data migration from previously conducted surveys of similar size and complexity? Will they support you on this front? Can the previous data be actively integrated in the new reports? Also ask them about the process that they take to ensure the integrity of historical data comparisons. This is relevant when the survey items are slightly different or if the previous vendor used different response scales. Considering potential organisational changes since your previous survey, how would the vendor work with you to determine accurate organisational mapping for the reporting of precise historical comparisons? How would historical results be incorporated into the reports? How long can you store data from prior surveys to provide comparisons for future use? Finally, you need to be very clear about the ownership of the data. Some vendors are very reluctant to release their survey datafiles to their clients’ prospective new vendors. Some may charge astronomical ‘administration fees’ to transfer the data to a new vendor. Others may claim their survey items or the combination of these is copyrighted and cannot be used by other vendors. The ownership of the employee data needs to be clear from the start.

15.  Online data capture: Make sure your vendor’s servers can handle large amount of survey users. Some vendors might be geared for smaller organisations, not being able to handle populations of 10,000 or more employees. Questions that you may want to ask on this front include: How many users can their servers handle? What is the vendor’s process for setting up and testing online surveys? Can the vendor’s system allow for unrestricted seamless branching of multiple unique survey versions? What different types of response scales does the vendor’s online system support (e.g., NPS, scaled items, open-ended items, ranking, etc.)? What support does the vendor provide for users during online administration (email or telephone help-desk for answering questions, contacts for technical assistance, languages, etc.)? Ask them about the process they use to audit responses and identify outliers in data samples. How do they send follow-up notices if employee response rates are low? User experience is also extremely important in employee surveys. The ‘look & feel’ and ‘functionality’ of the online survey or data capture tool will determine the success of your project. It is simple, poor user experience will lead to poor / low response rates. Some examples of poor user experience include inability to allow for your full corporate branding and colours, horrible standard landing pages with awful colours and poorly chosen images or even tacky cartoons, poor compatibility with mobile devices, very slow responding survey site, survey site that logs the user off after a few minutes of inactivity, impossibility to continue with the survey at a later stage, poor page layout (a survey that contains too many questions or too many open questions per page), lack of completion bar indicator, among others, are just a few examples of poor survey user experience. You need to ask your survey provider how they ensure high levels of employee survey participation. It is a good idea to request demos of their various online survey options.  

16.  Paper data capture: This is what makes the biggest difference between amateur and truly experienced employee survey providers: the vendor’s capacity and experience in administering paper-based surveys for clients. If you plan to administer paper surveys, you need to ask your vendor to describe this process and the steps taken to maintain quality control. The strongest vendors will be able to advise your organisation on how to best manage the paper survey administration process: booklet design, printing, shipment, distribution and collection of paper questionnaires, scanning of completed surveys, merging paper with online data, etc. Some questions to bear in mind: Do they typeset and print the surveys in-house or work with third parties to do so? Do they print and ship surveys from one central location or regionally? Do they expect you to do the printing and distribution of paper surveys to all your different locations? Ask them to explain their methods for paper administration sessions—do they recommend group administration sessions or at-desk administrations of paper surveys? Can they provide training and administration scripts for your local survey coordinators? How do they collect paper surveys while still ensuring respondent confidence that results are kept confidential? How do your local survey coordinators are to return completed surveys back to the vendor (i.e. business reply envelopes)? What experience does the vendor have in delivering multi-lingual paper-based surveys globally? Where are their scanning centres located? Before embarking in a complex paper survey exercise, ask your vendor if they can support you in administering your survey 100% online. No need to say that in certain sectors 100% online surveys are not feasible. 

17.  Reporting capabilities: Accurate reporting of collected data is of the utmost importance within the survey process. Your final reports should be easy to use, easy to access and easy to act upon. Nowadays, with the advances of technology, employee surveys should offer results in real-time. Some vendors tend to deliver their survey results two to three weeks after the closure of their surveys, by the time these results are cascaded to the lower levels of the organisation, six to eight weeks could have passed by, and of course, by that time some of these results may not be relevant any longer. When should you get the results to employees? As soon as you can. A good timeframe for this is no later than 20 days after the closure of the survey. Does your vendor have the ability to generate interactive online reports so that the report owner can easily navigate their results, identify the most critical issues and easily export results into different presentation formats (i.e. PowerPoint)? Does the reporting tool allow the user data filtering facilities, cross-tabulation functionalities, and multiple comparison of data to various benchmarks simultaneously? Does the reporting tool allow for multiple online data presentation choices and formats? Does it contain algorithms to automatically calculate drivers of engagement, retention, and the like? Does the vendor cater for busy (or unsophisticated) managers, those without internet access or low computer literacy, by also providing alternative reporting formats (i.e. Excel or PDF reports)? Discuss what options are available both in terms of content (e.g., graphics and data layout) and format (e.g., PDF, PowerPoint, Excel)? How would the vendor work with you to determine the format of the reports? Make sure you request examples, online demos and static reporting options – ensure you get the chance to try the tool with your employee survey team. In terms of data validity, how does your vendor review data to ensure that the desired response rate has been met and that responses are sufficient to provide for valid analysis? How does your vendor indicate in reports, especially for small teams, which differences between compared groups are significant so that management can be confident their HR decisions are based on reliable differences? How does the vendor process and report open-ended questions? Does it offer word-clouds? Sentiment analysis? Automated clustering of HR themes? Do they include the editing of employee comments and automated translation of all comments into a single language?  

18.  Data security: Your vendor must be able to offer adaptable, effective and secure online surveys and employee data handling processes. It is important to verify your prospective vendor’s commitment to data security. Make sure you understand your vendor’s information security management system, the framework of policies and procedures that includes all legal, physical and technical controls involved in their information risk management processes. Are they compliant with ISO 27001, the internationally recognised standard for information security systems? What sort of data security management/encryption is available? Ask them to describe how the data is stored and backed-up within their facilities or at a remote location. Where are their data centres based? Do they have capabilities for secure (https/SSL), encrypted questionnaires and report delivery? Permanent malware scanning? Latest security updates applied? Daily vulnerability scans? Daily backups? User password policies? How does the vendor ensure that survey responses are kept confidential? What options are available to control access to the online survey (i.e., password protocols)? You simply can't be too careful when it comes to employee survey information security. Protecting your employees’ personal records is critical – and a legal requirement. 

19.  Post-survey support: When employees are communicated their survey results, but nothing further is done with the data/no actions take place after the survey, this sends a very ‘negative’ message to employees: ‘Management does not care about your feedback’. A good vendor will always challenge you, they will take you out of your comfort zone, will tell you some truths about your HR processes and how to improve them. You need to verify if your survey provider has the expertise and capabilities to offer post-survey action planning support and advice. Can the vendor provide automated online action planning platforms with embedded libraries of best practice suggestions, online distribution of reports, communications templates, action planning guides, action planning workshops, bespoke improvement/training programmes, etc. to promote consistency in the survey follow-up process and ensure organisational change? 

20.  Transparency of fees: Make sure your vendor’s fees are transparent and linked to the number of employees that are to be surveyed or the complexity of your reporting requirements (based on your organisational structure). In some instances, HR colleagues told me that sometimes the initial core fee was acceptable and hence they agreed to sign the contract, but later learned of several additional hidden charges for necessary services that should have been made clear from the beginning and comprised in the original core fee. Some of these charges included survey results’ previews and presentations, additional reporting functionality, fees for changes in the survey content, fees for necessary data analyses (i.e. key driver and linkage analyses, qualitative analysis) and additional benchmarks. At the end of the project clients realised that they had paid nearly twice the originally agreed fee. Where possible, get three detailed quotes for the work to be done, and try to agree an all-inclusive fixed fee for the whole project to avoid hidden charges. Beware of annual survey subscription fees, these are solely for purchasing user licence agreements for specific software that may not include any sort of support (i.e. project management, advice on survey design and communications, etc.). This advice should save you some difficult moments renegotiating necessary project-related services for the already agreed fees.

Based on the 20 themes that we have just discussed, you may have noticed that I recommend considerable caution and care when selecting your next employee survey provider. This questioning approach is intended to make sure your purchasing transactions are transparent, methodical, comparable, sanitised, fair and free from any sort of external pressure or biased influences.

You think this is excessive? Not at all, I just want to make sure that I give you a heads up, based on my experience, about how potential lack of vendor selection approaches could defeat your employee survey initiative from the start. The intention is to guide you as the buyer and make sure you make an informed decision when selecting your next survey vendor. Remember we are looking at a long-term partnership here, so you need to choose the right partner, especially for implementing large scale global projects… the cost of failure could be high!

However, having said that, do make sure you keep your RFP as simple and straight forward as possible to encourage competition and completion. Remember, do ask the right questions if you are to find the right answers. I wish you good luck with your next employee survey…

Thanks for reading my blog. Were any of the insights provided of value to you? I would welcome your feedback - please do ‘Like’ or ‘Comment’ your experience on this subject in the space provided!

Thanks for your kind support - Follow me on LinkedIn for more articles and insights!

Disclaimer: The author is making this ‘Opinion Blog’ available in his personal effort to advance the understanding of best practices in workplace related matters. The author assumes no responsibility or liability for any errors or omissions in the content of this ‘Opinion Blog’ or for the results obtained from the use of the information provided. The information is provided on an ‘as is’ basis with no guarantees of completeness, accuracy, usefulness or timeliness and without any warranties of any kind whatsoever, express or implied. The views expressed are solely those of the author in his private capacity and do not in any way represent the views of any entity whatsoever with which the author has been, is now, or is to be affiliated in the future.