Given the fact that technology is changing so rapidly, all organisations understand the importance of investing in continual learning, so that they can remain on the cutting edge. Now it’s all very well to talk about the importance of being a learning organisation, but how do you actually measure the learning of your employees?
This can be a major challenge, especially in the new technical areas, such as NLP and artificial intelligence. How do you know whether your programmers and computer engineers have the skill sets needed to compete? Do they need additional training? Which is the best training method? And whom do you select for advanced training? Or should you hire fresh blood?
These are extremely important questions which keep CEOs, CTOs , delivery chiefs and business unit heads up at night. Surprisingly, they don’t know how to answer them, because they don’t have a good way of objectively assessing the technical skills of their programmers. This is a remarkable lacuna, given the fact that the success of these companies is based on the technical competence of their employees.
The Problem With Learning Assessments
They are hampered by the fact that technical learning assessments today are imprecise and inaccurate. They are usually very subjective, and are based on what the manager of that particular programmer feels about his skills. The technical competence of all the programmers in a team is not going to be the same, and it’s quite short-sighted to pretend that they are all equal. However, employee ratings are easy to game, and it’s not possible to know if an individual gets a high technical competence rating because he really is a programming whiz kid, or because he’s good at sucking up to his manager.
A star programmer can be 10 times as productive as an ordinary one. How do we identify these star programmers? How do we help them to get better and better? How do we get rid of the non-performing programmers who are just occupying space because they are better at buttering up their boss?
The problem with technical competency assessments today is that they can easily be gamed. Most technical skills assessment engines today offer only simple multiple choice questions. These are notoriously unreliable because a smart programmer can guess the right answers, even though he is completely clueless about the content. Another commonly used evaluation option tests algorithmic skills, but this is useful only for employees whose job involves a lot of coding. It’s pointless testing this for the umpteen other roles (front end, back end) where these skills are not related to their day to day work.
The gold standard is to test using real life coding problems which the engineer has to solve so he can prove his practical capabilities. Now, this is typically done using a whiteboard with a highly skilled engineer taking an interview. However, because this requires human intervention, it is very expensive and time-consuming and it is not possible to apply this for junior hires. This is where the problem starts because mark sheets and grades from India’s colleges are highly unreliable.
We need a better method for technical assessment training – an engine which is human-independent, and which also encourages continuous learning. This would be very useful for the company because they can fast-track their stars and assign them to the best projects, so that they’re able to create more wins for the company. They will also be able to identify the poor performers, so they can upskill them and reskill them, to make sure they don’t get left behind.
It’s only if the company understands the technical strengths and weaknesses of its people at an individual level will it be able to do this efficiently. These people analytics can be used to ensure that employees have the cutting edge skills the company needs to remain relevant as the world evolves.
These metrics would help the business delivery heads to plan their training courses, because they could see what their skill gaps are, and how fast they need to fill them. This objective criterion of competence would also help business managers to staff their teams more intelligently, so they can optimise their employee utilisation scores. Right now, they wing it, and get the smart guys to compensate for the incompetent coders, but these metrics would help them to make better assignment decisions.
What Chief Human Resources Officers Are Doing
Chief human-resources officers are already starting to deploy predictive models that can help them to identify, recruit, develop, and retain the right people, so they can maximise their human capital. Mapping HR data helps organisations identify pain points and prioritize future investments.
DoSelect has come up with a clever, cutting edge online solution for providing analytics on the technical competence of employees. It has created simulated assessments which rank candidates on the skills which are relevant to their daily tasks – not just algorithms.
This is very helpful for the employee as well, because he can objectively understand what his deficiencies are, and then ask for help to fill these in. He will be able to accelerate his own career progression – and will be able to objectively prove that his skills are improving at the time of his appraisals.
This will create a positive virtue cycle, as ambitious programmers will compete with each other to complete the assessments successfully, in order to get raises and promotions. Just like doctors need continuing medical education (CME), computer programmers need continuing technical education as well. HR understands the importance of this, but most HR people are not qualified enough to be able to judge the technical skills of the coders.
Meritocracy Should Win
A dashboard which displays the technical competence of a programmer over his career will allow the organisation to become a true meritocracy, where the good engineers get promoted because they’re technically better. This is the kind of company which will attract even more good engineers because they can see that their career progression can be meteoric if they are skillful.
A company which is able to measure technical skills properly, and reward them well, has an unfair competitive edge in the market because it will quickly become the employer of choice!
Even better, this dashboard can be used to prove the value of the training which the HR dept is tasked with providing. All IT organizations spend gobs of money on training and many have even set up their own universities.
Sadly, they don’t have any idea as to whether the training works; why it works; when it works; which training works better and for whom it works, because there are so many variables which they cannot parse because they do not have the data analytics to do so (which is a bit of an irony, given the fact that these IT organisations make money by providing data analytics for their clients!)
These companies still can’t answer basic questions about the training they offer. Is it better to do the training online? Is peer-to-peer better? Is classroom training better? Should it be done informally, one on one? There are lots of options, but this just generates a lot of confusion, because the feedback loop never gets closed.
The problem is that companies today spend millions on technical training, but they are hamstrung because they have inadequate measurement tools to assess the results of their training courses. Today, all they have to rely on is participation and completion data and employee feedback reviews, but they really don’t have a way of being able to measure improvement in technical performance in a quantifiable way. This is the holy grail, and this is what the DoSelect (a company in which I am an angel investor) dashboard promises to offer.
What DoSelect Does
A major pain point today is assigning engineers with the right skills to the right project. This is a key decision because they need to make sure they match the engineer’s competence with the complexity of the project. Today, the team leads and project managers work closely with the HR teams to identify valid profiles.
They query the HRMS/HRIS systems to identify people with relevant skills, and then use those shortlisted profiles who are then interviewed by the internal tech team, and then by the client. The problem is that the talent pool has become so shallow, that the majority of profiles are inadequate. The company then either needs to hire new employees on a war footing, or re-skill their employees – both of which are complex and expensive exercises.
They have been able to get away with these inefficiencies for so because they had very fat profit margins. However, as these get whittled, they are going to have to use better tools to measure the technical acumen of their engineers more accurately.
Assessments are not the end but the means to an end – the end being data driven decision making in personnel management. We need to treat people as people – not as widgets.
[This post by Dr. Aniruddha Malpani first appeared on LinkedIn and has been reproduced with permission.]