Museum Immersive Experience

I spearheaded the creation of an Augmented Reality application for the conference, strategically aimed at both enticing prospective clients and propelling our company into the burgeoning XR domain.

Key Contributions

Ideation, Designs, AR Prototype


Team

PM, Tech Lead, Developer & me

Tools

Adobe Aero, Generative AI, Figma, Skybox, Bezi

Timeline

2 Months
July '23 - Aug '23

INtro

Stepping into the XR world.

Stepping into the XR world.

The world is always changing and the tech world in particular is always advancing. As a consultant company, that means we need to be at the forefront of new emerging technologies.

Thoughtworks had a hackathon to generate ideas for a better and more modern museum experience to attract the younger market. I participated with a team and was able to design 2 Virtual reality spaces and an augmented reality filter which landed me a role to help out in the Museum and potentially showcase this to in the Google Next Conference.

GOAL

What can we develop that not only attracts
clients but also leverages our current
partnership effectively?

Our team wanted to create an app we can re-use for a client, so we came up with a number of ideas to attract clients but also be able to pitch to a current client.

One aspect of the app we kept was it being an Augmented Reality app. Our company wanted to go into Experience Reality field and we knew that an AR app can attract all kinds of clients in retail, medical, museums, and much more.

I drew inspiration from Coach's successful pop-up shops, where they innovatively utilized Augmented Reality to showcase wearable purses and wings, effectively promoting their latest purse collection

Another idea was to make an "unstill" image, when users would scan their phone on an image, in a sense it would "come to life" and animate.


LIMITATIONS

Budget and time constraints

I was onboarded to this project in mid July 2023 and we had to submit the project by the mid-August for approval. There was only a few weeks for us to produce a project and the budget for our project was under $100.

Additionally, the PM and Tech Lead were occupied with another project, placing time limitations on their involvement. They relied on me to design the prototype and coordinate with the developer in China.

Some challenges I had to face:

  • Initiative in Designing for an Augmented Reality Prototype

  • Collaborating with Eastern Time  and China Time zones, resulting in 4am meetings

  • Can't rely on finding animators with budget and time constraints

  • First time using Adobe Aero and Generative AI

PROJECT GOALS

01

Create an app that can be reused for Museum


02

Incorporate Augmented Reality


03

Ready to showcase at the Google Next booth

IDEAS AND BRAINSTORMING

Learning about Lincoln

With our partnership with the museum, we wanted to focus on how we can make learning about Lincoln more interactive. We had a few ideas of having Lincoln giving the Gettysburg Address and adding sounds of his speech. Also, we knew we wanted an AR app, so we had to consider the users holding up either their personal device or tablet installed at the booth.

Sketches

AI animation for Ideas

Google Next
Conference
 Booth

Our booth would be 10x10 ft and there would a crowd of people we had to take account while designing

Design

Storyboards

With our partnership with the museum, we wanted to focus on how we can make learning about Lincoln more interactive. We had a few ideas of having Lincoln giving the Gettysburg Address and adding sounds of his speech. Also, we knew we wanted an AR app, so we had to consider the users holding up either their personal device or tablet installed at the booth.

Design

Figma Prototype

PROTOTYPING

AR Prototypes

I realized that Figma is limiting and in order to communicate to the developers, considering the language barrier as well, what I was envisioning, a 2D prototype would not be enough. I decided to take my iPad and cut out images through Adobe Photoshop and transfer the images to Adobe Aero.

Learning Curve with Adobe Aero

When I first started using Adobe Aero, I just placed the cut-out images of lincoln and added clouds with animations. I tried my best with laying the images so that it can bring the "wow" factor.

Changing the Image

Here we decided to take another image to portray the Gettysburg address and add more elements like the graveyard site to the the AR prototype.

Using Anchors

Since we wanted users to scan the image and have it display an AR prototype, I decided to print the image and anchor the elements to the image. I also added the speech in the middle and had buttons so that users can go back and forth to read the speech.

Learning Curve with Adobe Aero

When I first started using Adobe Aero, I just placed the cut-out images of lincoln and added clouds with animations. I tried my best with laying the images so that it can bring the "wow" factor.

Changing Lincoln's Poses

Since we couldn't have the budget or time to find an animator for Lincoln, I decided to re-draw Lincoln in Adobe Photoshop and my iPad into different poses with his hands and month for each part of his speech.

Changing Lincoln's Poses

REDESIGN

Last Minute Changes

The image that we were using got disapproved by the conference committee, so we had to go back to the drawing board to come up with another image.

After submitting a few images we thought would have historical background and can be used for the museum's app, we got approved for the image below of Lincoln discussing the Emancipation Proclamation with his Cabinet members.

A big challenge I faced with this image was the layering aspect of this. There was no way to cut out and add image of each character as I did in the previous prototype because it had way too many details and the AR prototype would look messy. I wasn't an artist and we only had 2 weeks to figure this out.

So, I decided to take a change with Generative AI.

AI is useful, but stupid.

There was a learning curve I had to face with Generative AI, it took me days to get the image I wanted from Adobe AI, but I managed to get the characters full body behind the desk and it looked pretty convincing to me!

Getting close

Successful Prototype & Approved

AR + AI

Thanks to Generative AI, I achieved significant cost savings by bypassing the need to hire an online animator or sketch artist. I seamlessly extracted images of Lincoln and the cabinet members, skillfully layering them with the table to craft a captivating AR experience. I was able to send this to the Developer so he can launch this on Android as an app and iOS devices as test flight app.

Our supervisors loved this prototype when presented to them, so that was a success!

NEXT STEPS

Make it interactive and learning

Even though the initial goal was to get it turned into a "unstill image" it felt incomplete. I pushed for a learning opportunity where the user can click on a person and they can tell their honest thoughts of the situation at the time. This way of story telling will make it fun and memorable for the younger generations.

The project was put on hold for now and unfortunately I got laid off right after this, but I plan to make my own version in my personal time!

IMPACT

Final Thoughts + Next Steps

Embarking on this project has been an enlightening journey, introducing me to cutting-edge technologies that were entirely unfamiliar to me before, such as Adobe's generative fill AI and advanced AR prototyping tools. The hands-on experience with these innovative tools has not only broadened my skill set but has also significantly accelerated my learning curve in the realm of emerging technologies.

What makes this experience even more fulfilling is the fact that my company recognized the value of the prototype developed during this project, choosing to showcase it in their Google Cloud booth. This not only underscores the project's success but also marks a significant milestone in my professional growth. The exposure to these new technologies, coupled with the recognition from my company, has left me immensely gratified and eager to continue exploring and contributing to the dynamic landscape of technology.

Key Contributions

User Research, User Flows, Hi & Lo-Fidelity Prototypes

Team

PM, 2 Engineers, and Myself

Tools

Figma, Miro, Trello


Timeline

2 Months
Dec 2020 - Feb 2021

Scott Wueschinski

Technical Director

Thoughtworks

Ayesha's extensive experience in product design, storytelling, and content visualization made her an invaluable asset to our team. Her background in designing mobile and web apps and her passion for mixed reality gave us unique insights and innovative solutions.