The 100% A Strategy - A Data Analysis of SPM 2023 with Python and ChatGPT 4o

In case you are from Malaysia or you have been near any Malaysians of late, you may know that in Malaysia, we have a national exam called the Sijil Pelajaran Malaysia (SPM) that is taken at the end of high school.

SPM data analyses have been super popular in Malaysia recently, partly popularized by the incredible Cambridge-trained economist and data scientist Thevesh ., who's been sharing his very cool visualizations of SPM data on both X and also LinkedIn ever since SPM results were released on 27th May, 2024.

Curious about his work, I dropped Thevesh a quick message, and he was kind enough to share with me a little bit about what he had done - and I found myself analysing a little SPM dataset from Lembaga Peperiksaan (Malaysia's Examinations Board) that Thevesh was kind enough to send to me through Github, which I did with a mixture of Python ala Anaconda and OpenAI's ChatGPT 4o in data analysis mode as I dusted off my economics training.

What it told me was what I now call "The 100% Strategy".

If you want 10A's for SPM guaranteed(?), you should take Apresiasi Tari (Dance Appreciation), Sinografi (Sinography), Tarian (Dance), Aural dan Teori (Aural and Theory), Produksi Seni Persembahan (Performance Arts Production), Reka Bentuk Industri (Industrial Design), Multimedia Kreatif (Creative Multimedia), Sejarah dan Pengurusan Seni (History and Management of the Arts), Lakonan (Acting), and Penulisan Skrip (Scriptwriting).

Why do I say that right there?

We will talk about correlation v. causation a little later (of course!), but if you look inside the SPM dataset for 2023, you will notice that for each of these ten subjects, the proportion of A grades is 100%.

What does this mean?

Every single candidate who took these subjects in 2023 received an A for the subjects, no questions asked.

Here is a handy visualisation to help you see what I mean, with the proportion of A's for SPM 2023 sorted in descending order of proportion and number of candidates.

...And here is a small table containing the sorted data.

Yes, these are real subjects, and yes you can sign up for them as a private candidate on the Examinations Board website in case you want to relive your SPM years for some odd reason, or force your children to do so.

In other words, it is not irrational to choose to take these subjects in the hope of getting A's for them, although I would caveat that this should be interpreted with caution and past results do not indicate the future!

Anyway, you might well ask:

Why is this relevant at all?

This is relevant because whether we like it or not, we do live in a grade-conscious society and grades are, again whether we like it or not, important for determining whether certain outcomes take place or not.

More concretely, if you're from Malaysia and you've been through a government school of any sort, you probably know that scholarship bodies and universities care about the number of A-pluses that you get in your exams and make university placement and scholarship offers on the basis of SPM grades, commonly the number of A's that a student has.

It's not that it's illogical - but it's certainly the case that at the highest levels of competition, this does result in a drive towards the top where people try to get the largest number of A's and A-pluses that they possibly can, find anything in their power to get ahead of the competition.

As I reflected on this and also the realisation that life is about much more than just grades, I realized that it was easy to be blinded - yet, recognising that this was part of reality, it made me wonder:

If I were to go back in time and re-do SPM with the aim of getting a scholarship from a scholarship body that simply requires me to get a certain number of A's, what should I do?

Certainly, many scholarship bodies require A+'s rather than just A's - unfortunately, Lembaga Peperiksaan doesn't really break down what kind of A it is that has been received right here but hey, an A is an A, and I figured that it was interesting to discover the answer to that question - and here it is, for your consideration:

If you want 10A's and you're making your judgment just based on historical data, take Apresiasi Tari, Sinografi, Tarian, Aural dan Teori, Produksi Seni Persembahan, Reka Bentuk Industri, Multimedia Kreatif, Sejarah dan Pengurusan Seni, Lakonan, and Penulisan Skrip.

Now, the reality is that in order to receive an SPM certificate, you still need to take certain basic subjects such as Malay, History, Math, English, and so on so forth, and so even if you were to try to game the system, you'd still find yourself in a position where if you don't study at all, you'll find yourself in a situation where you have 10 A's and also maybe 5 G's or something of that nature.

...But maybe you don't care?

That is entirely your right!

Even so, before you start enrolling yourself in some of these rather... Unique subjects, it's time to throw some cold water on that hype, as we consider...

What explains a 100% A grade rate?

Intuitively, a 100% A rate suggests that a subject must be 'easy' to get an A in, and therefore that taking the subject entails that you will do well - one reason you might be hovering your cursor over that registration button at the moment.

Even so, simply because it's not irrational to choose to take Dance Appreciation (hey, it could be interesting!), not all that is rational is correct, and there are some things worth thinking about.

#1: Self-Selection.

In the SPM 2023, the subject with the smallest number of students was Reka Bentuk Kraf with 17 students.

The proportion of students in Reka Bentuk Kraf relative to the 372,656 students who took the compilsory Sejarah (History) in 2023, is approximately 0.0046%, which tells us that not many people elected to take Reka Bentuk Kraf relative to the total population size of SPM takers, to say the least.

This was true for all of the other subjects for which there was a 100% A rate, with Reka Bentuk Kraf only having 17 students, Sejarah dan Pengurusan Seni having 67 students, Produksi Seni Persembahan having 171 students, Multimedia Kreatif having 23 students, Aural dan Teori having 71 students, Penulisan Skrip having 100 students, Apresiasi Tari having 56 students, Lakonan having 44 students, Tarian having 56 students, Sinografi having 44 students, and Muzik Komputer having 76 students.

Please bear in mind, therefore, that the subjects identified are very niche subjects and they often require specialised equipment or training, and secondly, bear in mind that there are very, very few candidates for every single one of the subjects identified here.

Given that we have a maximum of only 171 candidates for any individual one of these subjects, we are forced to consider why that is the case, and it leads us away from the "they got A's for this subject because it is easy" interpretation - indeed, if you are one of the few students who took these subjects, there's a good chance that you were interested in the subject and invested enough to receive training, practice, and coaching to participate.

In other words, one possible interpretation is that if you were one of the rare people who took these subjects, chances are that you had been quite interested in the subject itself, hence why you'd make a special accommodation, and therefore why you might be likely to get an A for the subject.

In other words? Correlation does not equal causation.

Simply because 100% of the candidates for all the subjects I identified got A's for each of the subjects I identified, there's no guarantee that taking these subjects will definitely grant you an A, and there is no guarantee that you will get an A in SPM 2024 or 2025 simply on the basis of this data because the candidates may have gotten A's on the basis of their interest, ability, and disposition - NOT because the subject is easy.

If you just go right ahead and take all of these different subjects but you have no interest in them, that could still mean that you just don't end up doing well for it because unlike the rest of the people who took it, you just didn't have the interest or nature to do well.

That said, let's look at the next interpretation.

#2: Coursework basis.

None of these subjects are listed anywhere in the SPM timetable.

Now before people out there start screaming about how there must be some scamming and cheating going on, know that they are all coursework subjects.

What that means is that essentially in order to receive a grade, you submit a portfolio and coursework over to the examinations board, and then from there you receive a grade.

Of course, if the percentage of A's for the coursework is to the point that every single grade is an A, then that might raise some questions about artificially inflated standards, but that's not something that I'm about to comment upon too extensively, partly because that can be accounted for by the self-selection I talked about a little bit earlier on, namely that only people who are interested in these disciplines might be fascinated enough to undertake the long hours of research for the purpose of performing well in these exams.

Still, is there something afoot? That's a separate investigation to be made, and I leave it out there to those of you who are interested enough.

What I will say is, coursework isn't something to shake a finger at right there, it does require commitment and skill to be able to create a coursework project, and even more so if it's something that you're not really a fan of.

Is it worthwhile to investigate whether there's a scandal such that every single person who takes the subjects I mentioned will receive an A regardless of how poorly they do?

I suppose that's up to you to decide!

Now, what do we make of this?

The data reveals to us that there are ten SPM subjects in which the A rate was 100%, and I pointed out that there were several possible explanations to this that elude the easy intuition that if you want 10A's, you can go ahead and register for every single one.

Of course, someone might very well read this and decide to register for all of those subjects at the same time anyway in the name of using this data to make an informed decision.

To them, I wish all the best - I am not here to judge, only to present nuance, to play Devil's advocate, and therefore to ask:

If it were really possible to get 10A's in SPM or any other form of consequential public examination just by following this advice alone, should you do it?

I suppose some people might, in the push for scholarships, university admissions, and any number of other things in which the question of exam results are important - and frankly, it is hard to blame them if they don't particularly care what subjects they take, the people evaluating them don't care either, and their only goal is just to get to a better place in life.

To each their own utility function, of course - but if nothing else, perhaps what this may tell us is this.

Whether we are a scholarship body looking for the best students, a selective employer seeking after the best talent or otherwise...

Perhaps we shouldn't be looking at people just based on the number of A's that they have. Rather, perhaps we need to look more closely and beyond the superficial in order to get a real sense of what people actually are about - their character, their motivation, and their purpose, before we make consequential decisions like whom to hire, whom to choose for a selective university program, and whom we choose to represent us.

This is, after all, something we aren't insulated from this even if we are looking at more 'difficult' exams, however difficulty is operationalised. (Failure rate? Proportion of grades below C? Dropout rates?)

Although exam results measure something and they are certainly important particularly with respect to certain professional qualifications and certifications, they aren't the only thing that we should consider, and evaluating a person requires a deeper insight that simply a gaze upon the surface, at the report cards, or a blind deference to what the late Daniel Kahneman would have called System 1 thinking - rapid, intuitive, but in the absence of verification in the face of a reality to which intuition does not apply, potentially hazardous and externally invalid.

Whatever the case though...

Enough rambling!

You now know the 100% strategy, its ins and its outs. If signing up your child for SPM Dance Appreciation as a private candidate is something that you want to do right there, please let me know how that goes - Do leave me a message, leave me a comment, and I would love to hear your story.

On the other hand, if you just enjoyed this analysis and would like to hear my thoughts on education and exam performance, or watch the (many!) conversations that I am fortunate to have with some of Malaysia's best thinkers, make sure to follow me on LinkedIn and YouTube for more in the days to come!

Alright, and that's it, thanks for reading, and I will look forward to seeing you in the next one!

Yours,

Victor.