A tale of regression and regressiveness – Statistics

Mahima had been a bright performer at her school. However, with the passage of time, it started to seem as if she was flailing about in academics. She liked studying and always had been an enthusiastic learner, which was validated by her curiosity for subjects beyond the scope of her syllabus. In a similar vein, she was amazed by the vast body of knowledge in the form of books at the library and the internet-enabled computer, which seemed like a treasure chest to her.

Though, she was irked by the disproportionate bias that was attached to the so-called exams. In her pursuit of wisdom, she would often be reprimanded by her teachers for “not sticking to the syllabus” and by her parents, who thought she was a “recalcitrant, pampered child”. Her classmates’ unrestrained passion for outscoring one another at unit tests, periodicals, etc. made her question the unending race.

All of this made her rebellious and full of contempt. She had coined a term for these traits and the individuals exhibiting them: rabid regressive. She scribbled the term onto her bag, textbooks, and almost every other article that was a token of this dreaded connection.

The status quo was going to change though.

Mahima’s school often organized seminars with industry experts, bureaucrats, and actors, to encourage students to develop offbeat career interests. Mahima thought of them as a useful pastime, which gave her more food for thought.

On one such occasion, the school auditorium was hosting a bright, experienced female statistician, Nidhi. She was also an alumna of the institution, which made her all the more excited to host the gathering. She wanted to make a difference.

A couple of minutes had gone by before she could begin. “A very good afternoon to you all!” She introduced herself and talked briefly about her education as a statistician in a premier institution and her current stint as an analyst at a multinational firm.

“Whatever I just said wouldn’t have made a lot of sense to you. Nevertheless, it will if you stay with me. There was an incident back in the day that ensured I ended up pursuing this field. Starting sixth grade, we were constantly prodded to read more; while this explained why we had the dreaded library-reading period, it also enabled the ones taking it seriously to improve their vocabulary and overall fluency in English.”

The audience was piqued by the mention of a familiar phenomenon. Nidhi continued, “Anyway, we were introduced to data representation in the sixth standard and I was fascinated by it.”

“So, I started devoting a lot of time to the subject. Ma’am Amrita, our maths teacher, saw this and started encouraging me to learn more. I would plot whatever data I could get a hold of; most of the time ten of my best friends were the ones who furnished these numbers; it had indeed become a hobby!”

“During one such exercise, I stacked the number of books my friends had borrowed from the library and read during the last four half-yearly terms against their score in the English exams.” Mahima grimaced slightly but she was far too absorbed to stop listening.

“Do note that the scores were relative; as you may know, our report cards pegged our performance in comparison to the rest of the class. Also, our library exercise books had book reviews that recorded the lending we did. Thus, the data is perfectly non-fictitious,” she said with a chuckle. “What I observed can be seen on the screen.” A set of enthusiastic eyes followed the presentation pointer’s ray and saw a chart.

“What I got was intriguing; I went to Ms. Amrita, who introduced me to a new and amazing idea: regression. If you try fitting the points, the overall trend is envisaged better.”

Mahima was startled. The overall trend of the increasing scores, with their dependency on the number of books read was quite succinct. She also understood that the number of individuals sampled was ten itself, with the more enthusiastic readers getting brighter with time. Nidhi continued, “While this cannot be called a regression proper since the sample wasn’t representative, it explained the overall idea to a large extent, especially to an unacquainted individual”.

“A regression maps a causal relationship between variables. In this case, the ..” and Mahima’s conclusion followed as it is. “This was in fact an example of a linear regression, in which a dependent variable is influenced by an independent one; in this case, the two follow a linear relationship, wherein the increase in the number of books read transcends to an increase in the scores obtained.”

“With Ms. Amrita’s exhortation, I persisted in pursuing the field during my time at school. During my undergrad stats’ studies, I was able to build upon it holistically, which imparted me with truly powerful knowledge.”

“I ended up becoming an analyst as I had said; it’s a job profile where regression continues to come in handy. For instance, companies optimize their budget among the various media they utilize in order to advertise a particular product. I assisted my team with such a problem for our TV and social media channels to arrive at a solution which ensured optimal spending for customer acquisition.”

“The general idea being, we had to arrive at an optimum sum of the two variables, with one influencing acquisition significantly more than other. The main thing to note is the amount of variation of y with x, in absolute terms. We rely on computers to plot these since using graph papers isn’t feasible now”, and the gathering broke into laughter.

“Regression analysis forms an inseparable part of a wide array of other disciplines. It enables decision-making and finds its application in various domains like machine learning. In fact, I believe, not learning regression may make you regressive!” Mahima couldn’t resist herself when Nidhi concluded. She ran after her when she began leaving the auditorium.

“Ma’am, back when you were here, how did you face the rabid regressives?”, she squealed while huffing between deep breaths. “What do you mean dear?”, Nidhi inquired. There followed a hasty, drawn out explanation.

“Being an outlier is exceptional! Make the most of it!” Nidhi’s quip kept resounding in Mahima’s head. She began understanding that despite bucking the trend, she could continue to remain a part of the population. On Nidhi’s advice, she even began referring to online resources to pursue her newfound passion.

Nidhi told her “Make transformations, logarithmic and exponential.”

Just like you, she wondered what the latter meant.

Ayush Pareek

I am a programmer, who loves all things code. I have been writing about data science and other allied disciplines like machine learning and artificial intelligence ever since June 2021. You can check out my articles at pickl.ai/blog/author/ayushpareek/

I have been doing my undergrad in engineering at Jadavpur University since 2019. When not debugging issues, I can be found reading articles online that concern history, languages, and economics, among other topics. I can be reached on LinkedIn and via my email.