Introduction

Over the past 10 months, Streamoid and WebMonks have been partnering up on various computer vision research topics like image classification, object detection, single view 3D reconstruction etcetera. Along came Google’s AutoML, and the researchers amongst us feared that soon we might find ourselves without jobs. After all, AutoML promises to make Artificial Intelligence accessible to anyone by automating the training process of deep neural networks. In recent computer vision research, that’s a big part of the job. Is 2018 already the year where machines replace a highly educated workforce with PhD’s under their belt?

In this blog post we will explore the options of AutoML applied to classification of different type of dresses. We evaluate AutoML on both the ease of use as the results.

Training the model

Getting a model trained is effectively as easy as described in the picture above.

• Step 1 can easily be done following the documentation provided by Google. It however requires setting up a billing account and linking it to the project.
• For step 2 we had to resort to using Google Cloud Storage to import the images as uploading images is limited to 500 images only. All interactions with Google Cloud storage can happen from within the browser. But for more experience users can also be done with the command interface using the Google Cloud gsutil tool.
• Step 3 and 4 can be combined and could be called the most difficult step in this process. And even this can easily be done by providing a csv which links the uploaded images to the right label. So some programming experience is needed to fulfill this step.

• Step 5 is starting the model training which is as easy as pushing a button. The only options to choose between are a free basic model and an advanced paying (=550$) model. We only tested the the free basic model. Training was indeed done in less than an hour.

After training the model

AutoML provides some visual result as insights into how well the model is trained.

Getting predictions out of the model can only be done online. This can be done in the browser by uploading pictures but to predict on more than 10 pictures one has to resort to programmatically accessing the correct web service. Getting this right proved to be a challenge as documentation to get this working was scarce. This can be forgiven given that AutoML is still in alpha release.

Results of the model

We trained models with AutoML on the all the images that we have been using while training and validating our own custom models ( total = 5025 images). We evaluated on an independent test set with images never used during training or validation (total = 564 images). On this test set AutoML scored 80.5% accuracy while our best custom model managed to get accuracy up to 85.9%.

Conclusion

The basic free version of AutoML provides an easy and quick way to get a good working image classifier. Its accuracy, however, is still outperformed by building and training your own custom models. Our data science jobs remain safe for now. That said, AutoML proves itself a valuable tool for prototyping as speed of delivering is unmatched. It remains an open question how the advanced paying model matches up against our custom models.

References

1. https://cloud.google.com/automl
2. https://www.blog.google/topics/google-cloud/cloud-automl-making-ai-accessible-every-business/
3. https://techcrunch.com/2018/01/17/googles-automl-lets-you-train-custom-machine-learning-models-without-having-to-code/
4. https://9to5google.com/2018/01/17/google-cloud-automl-vision/