- A new dimension
Thanks to my college, I had an opportunity to learn about Machine learning and Deep learning concepts early on from my sophomore year of Bachelors's. I further explored this hot field through collaborative research on various applications through my internships in reputed organizations.
In this article, I want to discuss a possible way to look at Machine learning research to make it more interesting. Let us constrain our view to the supervised learning setup. This discussion could be extended to unsupervised too. With this, I want to dive into a possible direction of Machine learning research to make it enjoyable further.
While having a little chat with my colleagues on Machine learning, I heard views that it is an annoying task of tuning the parameters to fit a function (mathematically) that could maximize the accuracy, inherently reducing the loss. I have also heard views such as research in these domains are saturated, especially in natural language processing. While the harsh reality of Practical ML is indeed a tedious task of finding the optimal hyperparameters given dataset, a new research in a direction specified in this article can make it fun.
I believe machine learning right now is a black box trying to fit the best possible function "empirically" given a dataset. The way machine learning is estimating the mathematical function (of inputs) is by hiring the concepts from convex optimization. However, since real-life applications mostly comprise of non-convex settings that is devoid of global minima w.r.t loss function, we adopt an iterative approach to find local minima. While iteratively solving a mathematically constrained problem, we run into various unknowns (for instance, learning rates) that have to be initialized manually. Now it is like solving a mathematically constrained problem with many unknown variables. As a result, we could end up in an overfit or underfit model. A practical machine learning scientist's job is to ensure a proper fit of the machine learning model empirically for a given dataset, which could be irritating at times.
Nevertheless, here lies the challenge. Let us take the example of a compiler and programming languages. Decades back binary language was the only means to communicate with the computer, preferably a compilation of millions of transistors that could only understand 1's and 0's. With the advent of assembly language to C, followed by C++, compilers came into existence and evolved smarter to lessen human efforts. With programming languages such as python and Matlab (interpreted language), coding has become a simple task and fun. There has been significant research on transcompilers that translate from one high-level language to another. Natural language processing is applied here to make high-efficiency transcompilers possible[1]. Here come the innovation and research to make things simpler for people to use. I will not be surprised if computers will understand pure English in the coming years!
The challenge in machine learning is to make it more user-friendly and less annoying in terms of parameter tuning. The development of tools that could graphically interpret the effects of varying a parameter for given ML settings could considerably reduce the efforts. This tool will make hyperparameter tuning more fun and less tiresome. Optimizing the hyperparameters empirically cannot be avoided since it is dataset dependent; however, tools can be build based on mathematical concepts to make it more enjoyable. Machine learning enthusiasts can work on this aspect to make it a smarter machine learning!
[1] Lachaux, Marie-Anne, et al. "Unsupervised Translation of Programming Languages." arXiv preprint arXiv:2006.03511 (2020).