Is your ML Eco-friendly?

6 min readAug 15, 2022

Ok, perfect, today the world is immersed in the world of machine learning, deep learning and artificial intelligence. These disruptive technologies seek, on one hand, to maximize the income of large companies and corporations, and in the other hand, to have more human and social impact, objectives that in the end, should bethe most relevant under the premise of “AI to make a better world for future generations” or “AI to reduce the inequality gap”.

Every day new AI applications are discovered, machines that can detect objects better than humans, amazing autonomous cars driving from Tesla, natural language processing (NLP) technologies such as GPT3, which are capable to generate text almost like human, DALLE-2 that builds spectacular images only imaginable in our dreams, LAMBDA chatbot developed by Google that almost convinces us of machines with conscience.

Although everything achieved to date is impressive, all these applications are just the beginning. As professionals in this field every day we look for new and innovative AI applications, however, and before embarking on a new challenging project, we have to consider a term such as “the means to an end”, in other words, if we know how much of our project development can contribute to the emission of CO2. Do you know the energy consumed each time you train your machine learning algorithm?

With the cheapening of electronics for deep learning, GPU’s, CPU’s, RAM (on-premise), and also Cloud platforms such as AWS, GCP and Azure, the access to hardware for processing large volumes of data is within the reach of any mortal. It is very common to hear colleagues talking about acquiring the latest NVIDIA RTX GPU card to optimize their learning processes, or that if we choose cloud service, for example an ML Sagemaker AWS service with a 24GB GPU. In general are we optimizing our processs?, answer: resounding YES, we are more efficient but only in two aspects, in time and cost, but HOW? By “how” I mean if we have any idea of the impact of energy and CO2 emissions that we generate when training and tuning our models, in other words we do not quantify how our Machine Learning algorithms leave their carbon footprint.

Let’s see the following table, which shows the estimated training costs of the base models and their CO2 emissions.

Also take a look to the next table and its impressive amounts of parameters and training times.

From these data we can already get an idea of the energy consumption of these big models that today are being used by large companies on a daily basis.

Fortunately someone came up with the idea to measure the emission of our GPU’s through a calculator to understand how a simple training impacts our environment. For our example we can simulate with a popular GPU for deep learning, we are talking about an RTX 3080ti card, with a training time aprox. of 10 hours. The result metrics are totalized in the following table of equivalences: 1.51 of carbon emitted.

At first glance the numbers seem harmless but if we multiply the number of times we repeat our learning workflows(hyperparameter tuning), and also if we consider a “X” number of people at this precise moment they are pressing the “enter” button and starting their training epochs, the data collected is no longer so insignificant.

What can we do?

Use techniques such as transfer learning or pretrained models that are available.

As long as we can perform transfer learning or reuse a model pre-trained it is an advantage, it is not necessary to train from scratch.

Fine — Tuning vs Training from scratch

If you find a model that almost fits what you need, try adjusting its final parameters (fine-tuning) instead of training everything from scratch.

Start with small experiments and debugging

In case of having a large amount of data, before starting the training it is better to work on samples instead of the entire dataset. In this way any modification to our code that implies corrections, debugging, parameter adjustments can be done quickly without waiting for example 30 hours, finally when you are sure of all the parameters and code its OK start your training process with the entire dataset.

Review available literature to choose your parameters more wisely

Review and analyze all the available documentation to correctly choose the key hyperparameters for better adjustment to your model. Do not play in a crazy and random mode with all the parameters. Many times we randomly try different values for our hyperparameters, based on trial and error we adjust our model. It is preferable to have a better plan from which values are more relevant depending on our problem.

- Random search vs. grid search

These two techniques are good hyperparameter finding, but both can be computationally expensive. As far as possible try to use the random search which can give you better results without doing too many iterations.

Code Carbon Library

I have found a quite interesting library for python that with a few lines of code we can record all the energy consumption for each training process. This library is called CodeCarbon, which is a lightweight software package that integrates seamlessly with your Python code. This program estimates the amount of carbon dioxide (CO2) produced by cloud or personal computing resources used to run your code.

Installation

pip install codecarbon

Code example:

from codecarbon import EmissionsTrackertracker = EmissionsTracker(project_name="name",
                           output_dir="emissions/",
                           measure_power_secs=3600)tracker.start()# GPU intensive training codeemissions = tracker.stop()

Super easy, isn’t it?, Additionally I have built an available Streamlit application to have control over my projects, my trainings and how they impact the environment.

Basically the application is an extension of the codecarbon library, with a few lines of code described above we create a CSV file in the emissions folder for each training of each project. At the end of the training we insert the content of the file into a database wher a Streamlit App has access.

You can can check my app dashboard in this link.

At the moment this app is in an early stage of development, but it is a tool that helps me to control the carbon footprints generated by my ML projects.

Conclusions.

The artificial intelligence expansion has only just begun, every day this technology will continue to take over our routine lives, which translates into an exponential growth of new and innovative projects that will require to process larger amount of data. As engineers immersed in these technologies, we need to be more aware of our work and how affect our environment, we can contribute not only with the development of AI projects with social impact, but also to be more conscious the way we develop our work.

References

Is your ML Eco-friendly?

Written by Mauricio Arancibia