Virtual Worlds for Machine Learning

This course will take place as an online event. The link to the streaming platform will be provided to the registrants only.

Contents:

The Unreal Engine is one of the state-of-the-art 3D rendering engines, mainly used for game development. In recent years, however, its use in industry and science has been steadily increasing, which is further supported by new features from the producer Epic Games Inc. This course gives an in-depth training to using Unreal Engine as a data generator – by gaining measurements from virtual worlds. Using the ground truth data generated with a realistic rendering engine, projects gain more robust AI pipelines, insight into AI performance on quantifiable data, as well as measurements from virtual scenes with environmental conditions that can be manipulated. At the end of the course, participants have setup their own pipeline with UE and a simple ML workflow in one of the leading supercomputing centres.

Roadmap:

• Visualization pipelines with Unreal Engine
• Scalability, Generalization, Domain Visualization
• Using Pixel Streaming for Remote Visualization
• Introduction into WebRTC concepts, connectivity, and HPC usability
• Building an AI/ML pipeline from WebRTC
• Preparing the frameworks
• Parsing and using data
• Best practices

Prerequisites:

• Basic knowledge of Machine Learning frameworks – We assume that the participants are familiar with general concepts of machine learning and deep learning. For an introduction to these topics, we refer to open resources:
• The MIT introduction to Deep Learning Course (http://introtodeeplearning.com/)
• The Machine Learning course and Deep Learning specialization by Andrew Ng et al. at Stanford (https://cs230.stanford.edu/) and on Coursera (www.coursera.org)
• The notebook-based courses of fast.ai (www.fast.ai) and of Master Datascience Paris Saclay (https://github.com/m2dsupsdlclass/lectures-labs)
• Deep Learning, MIT Press book, Ian Goodfellow, YOshua Bengio, Aaron Courville (https://www.deeplearningbook.org/)
• This course should be visited with previous knowledge of Unreal Engine. While it provides some insight into how Unreal Engine work, it is not to be understood as an introduction. We refer to our in-house course “Introduction to Unreal Engine”, which typically takes place in March each year.

Target audience:

Scientists who want more robust computer vision pipelines or a concrete method of evaluation CV algorithms. We also aim at developers of analysis pipelines working on streaming data as well as developers who are creating and managing digital twin models.

Language:

This course is taught in English.

Duration:

2 days

Dates:

7 May 2024, 09:00-12:00, 13:00-16:00
8 May 2024, 09:00-12:00, 13:00-16:00

Venue:

Online via Zoom

Preparation:

To be well prepared, we ask you to install the Unreal Engine 5.1.* and a C++ Development IDE such as Visual Studio 2022 upfront. Details are available on https://go.fzj.de/unrealcourse, where we are updating information as problems and questions arise.

Number of Participants:

minimum 5, maximum 30

Instructor:

Dirk Helmrich, JSC

Registration:

Please register via the registration form until 23 April