Interviews With Xr Designers Simon Frubis Finishing His Masters In Xr Design

Name: Simon Frübis

Working At: AR/VR Lab of Otto-von-Guericke University Magdeburg Germany

Prior XR Designer Role: Started 2020 with my Master’s in XR so no prior XR Designer Role. Prior Designer Role: Head of Design and Co-Founder of a Digital Marketing Company.

How was your transition from traditional to XR design?

From 2014 until march 2021 I was head of design, co-founder and key account for clients like SAP. I was responsible for everything related to UI /UX design and on- and offline marketing. Most of the time I was working on client jobs, created designs for websites and built customized WordPress templates.

Why did you choose XR as your career move?

My heart beats for interaction design and everything around innovation in UI/UX. I see enormous potential in the AR to create more natural and emotional user experiences between humans and computers, especially with hand interactions and haptics.

Extended Realities allow us to break out of 2D screens and combine our real world with the infinite possibilities of the virtual world. Future XR creators will determine the rules of these virtual worlds. What could be more exciting to be part of this and explore new design patterns of the Metaverse. Besides GPU power, the only limit is one's own imagination.

End of 2020, in the middle of my master’s thesis and full of enthusiasm, I was faced with the question: Do I want to continue designing rectangles for rectangles or do I want to explore compelling natural user experiences in virtual worlds? Am I willing to take some risks, learn and start with something new to get a taste of the sweet fruit called XR? Hell yeah, it’s now or never! I knew that I had to go ALL-IN for this. My co-founder had different values and goals so I made one of my hardest decisions — I resigned and left my company for a career change. Since then, I am now fully committed to XR and the exploration of haptics.

How did you start learning about XR?

At the end of 2014, I did my first experiments with 2D markers in Junaio. One of the first mobile AR Apps for content creators. My goal was to find best-practices for stable 2D marker design.

Since then, I was more than convinced of the potential of AR. However, the technology was not yet ready and neither was I as a young student. In 2017 I invested my enthusiasm for AR into my bachelor's thesis. I investigated a concept for a mixed reality car configurator in urban showrooms. In 2020 I wrote my master's thesis about multisensory Feedback for Hand User Interfaces in XR.

This year was also the time I felt the XR industry was starting to get mature. I was ready to jump in at the deep end. I bought a Quest 2 on release day with a VR-ready Windows laptop. It was getting serious at that moment when I spent 2.500 € on a vr-ready development environment (my mac OS heart was bleeding).

The more I got into it, the more I realized how passionate I am about XR prototyping. I learned everything via Youtube Tutorials (special Thanks to Dilmer and his great XR Tutorials), endless googling and studying MRTK Docs. This was my first experiment with a custom hand UI made with the MRTK Framework.

I felt a bit alone as a designer in this field. However I saw more and more benefits of becoming an interaction designer with the ability to prototype your XR ideas. It's super fulfilling to know the right tools and which screws to turn in order to realize your vision.

In the beginning, I was still bothered by insecurities, inferiority complexes and influenced by the mindset “I’m a designer, I can't code”. But I trusted my vision and my willpower to achieve my goal one way or another. My passion for XR helped me a lot to take that perceived risk and overcome fears and some desperate holes of troubleshooting. I wanted to prove myself wrong and be an example that designers can overcome this fear and to be able to create playful and human centered experiences.

You need a different mindset for programming and debugging than you do for design. You have to learn to fail. Fail a lot! It's all very slow in the beginning. Don’t be too hard on yourself. Always keep in mind that you are just starting and every extra step is part of your journey. You’ll realize after your first working prototype, that it was absolutely worth jumping on that train to become an interaction designer for XR.

What was your first professional role in XR and how did you land it?

My master's thesis topic and a working prototype were my entry ticket to become a research assistant at an AR/VR Lab in Magdeburg. This gives me the opportunity to continue and improve my work for that topic I’m most passionate about.

Accelerate Your Launch Into an XR Design Career

Besides that it’s important to create awareness on what you’re doing. Post your progress clips on LinkedIn and Twitter. Be authentic, your followers want you not somebody else. And again, don't be too hard on yourself. Nobody judges you or expects polished prototype videos. Try to spread the message in short clips and respect the time of your followers.

How does your XR Design work look right now?

I am working alongside another interaction designers investigating XR hand interactions and scrolling interfaces on different surfaces. All the knowledge sharing and and project management is done over Notion.

For Ideation we are using FigJam or Miro boards in Figma, in Procreate and in Shapes XR Space. Interactive Prototyping is done in Unity. Everything from there is saved via git versioning on github

Here is the latest prototype for our first user study:

What was the hardest part and how did you overcome it?

I had a lack of basic coding experience in c# and working with Unity. I knew the basics of functions, if/else conditions etc. but I had to get familiar with the C# syntax + MRTK functions. Luckily for prototyping with MRTK you can already do a lot over the Unity Editor via drag and drop. To create your first custom interactive prototypes you just need basic coding knowledge (conditions, loops, functions) and an understanding of the framework. The huge MRTK example library helped me a lot to understand the mechanics behind it. I learned a lot by studying the docs, Youtube tutorials and trial and error. Here are some valuable links that helped me to get started with Unity and MRTK.

"I realized that I can’t apply 1:1 previous 2D design thinking methods for XR creations. I had to create my own workflow to create as quickly as possible valuable ideas. That’s actually the fun and exciting part."

Trying out new ways without any instructions. You don’t know if you’re going the right direction until you try it. A lot of Trial and Error. The most valuable methods for me have been Storyboarding, Bodystorming and Vision Work. In some of my videos I’m showing examples on how to apply these methods for XR design.

The Design Thinking methods are fundamentally the same. However, the focus and priority for some methods changes. Storyboarding and bodystorming, for example, are one of the most important methods to get a feeling for the vision in the right context.

Early work in VR e.g. with Shapes XR is one of the most important rules. For user testing, the same methods can be applied, such as 'thinking aloud' or qualitative interviews.

For some ideas you can’t do user research as there are no users yet. However you can create first prototypes and iterate through user testing. For this case there is also a method called Future Thinking. As I haven't practiced it yet, I will leave you with a good read and quote:

“Design thinking generates solutions based on today’s condition, but future thinking frames opportunities in a way that the present cannot achieve."

Your Favorite XR Tools and Suggestions

Day to Day:

• Software: Unity, MRTK, Sourcetree, Notion
• Hardware: Windows + Apple Laptop, Oculus Quest 2 with Handtracking IR 170 Cam by Ultraleap

Design Process:

• 2D: Figma, Figjam, Procreate (iPad)
• 3D/VR: ShapesXR

Previously Day to Day:

• InDesign, Illustrator, Photoshop, Adobe XD, Premiere Pro, HTML/CSS, Miro, Notion

Checkout my hardware setup and other XR design tools and frameworks.

If you gave one piece of advice to up and coming XR designers, what would it be?

I would definitely recommend the XR designer path. It’s an investment at the beginning but do it. You’ll be the experienced XR design veterans and early adopters in 10 years. Start to learn early and make mistakes. Train your mind to be kind to yourself on this journey. Maybe there will be better coders out there but there won’t be coders with your specific background. Every learning and skill is a building block that can guide you in unique directions where other coders would never go. See it as a privilege and use it.

There is so much to explore and we desperately need more interaction designers out there to design new working patterns for XR. When you check recent job offerings of big companies like Apple, Meta, Microsoft you’ll see that there is a big need for XR Prototypers and Interaction Designers.

Stop comparing yourself to others or finding reasons why you are not qualified for it. If you are reading this, chances are you already want it enough. So do the opposite! Write down why you should do it, what your vision and goal is for the first prototype and just do it!

Where Can People Find You?

• LinkedIn: Simon Frübis– Researcher and Prototyper for Hand User Interfaces in Extended Realities
• Twitter: @fruebis
• YouTube: Simon Frübis
• Discord: smon#1063

Save Your Spot: Interaction Design and Prototyping for XR