Tag Archives: Compose UI

Multimodal image attachment is now available for Gemini in Android Studio

Posted by Paris Hsu – Product Manager, Android Studio

At every stage of the development lifecycle, Gemini in Android Studio has become your AI-powered companion, making it easier to build high quality apps. We are excited to announce a significant expansion: Gemini in Android Studio now supports multimodal inputs, which lets you attach images directly to your prompts! This unlocks a wealth of new possibilities that improve team collaboration and UI development workflows.

You can try out this new feature by downloading the latest Android Studio canary. We’ve outlined a few use cases to try, but we’d love to hear what you think as we work through bringing this feature into future stable releases. Check it out:

Image attachment - a new dimension of interaction

We first previewed Gemini's multimodal capabilities at Google I/O 2024. This technology allows Gemini in Android Studio to understand simple wireframes, and transform them into working Jetpack Compose code.

You'll now find an image attachment icon in the Gemini chat window. Simply attach JPEG or PNG files to your prompts and watch Gemini understand and respond to visual information. We've observed that images with strong color contrasts yield the best results.

New “Attach Image File” icon in chat window
1.1 New “Attach Image File” icon in chat window

Example of multimodal response in chat
1.2 Example multimodal response in chat

We encourage you to experiment with various prompts and images. Here are a few compelling use cases to get you started:

    • Rapid UI prototyping and iteration: Convert a simple wireframe or high-fidelity mock of your app's UI into working code.
    • Diagram explanation and documentation: Gain deeper insights into complex architecture or data flow diagrams by having Gemini explain their components and relationships.
    • UI troubleshooting: Capture screenshots of UI bugs and ask Gemini for solutions.

Rapid UI prototyping and iteration

Gemini's multimodal support lets you convert visual designs into functional UI code. Simply upload your image and use a clear prompt. It works whether you're working from your own sketches or from a designer mockup.

Here’s an example prompt: "For this image provided, write Android Jetpack Compose code to make a screen that's as close to this image as possible. Make sure to include imports, use Material3, and document the code.” And then you can append any specific or additional instructions related to the image.

Example prompt: 'For this image provided, write Android Jetpack Compose code to make a screen that's as close to this image as possible. Make sure to include imports, use Material3, and document the code.'

Example of generating Compose code from high-fidelity mock using Gemini in Android Studio
2. Example of generating Compose code from high-fidelity mock using Gemini in Android Studio (code output)

For more complex UIs, refine your prompts to capture specific functionality. For instance, when converting a calculator mockup, adding "make the interactions and calculations work as you'd expect" results in a fully functional calculator:

Example prompt to convert a calculator mock up

Example of generating Compose code from high-fidelity mock using Gemini in Android Studio
3. Example of generating Compose code from wireframe via Gemini in Android Studio (code output)

Note: this feature provides an initial design scaffold. It’s a good “first draft” and your edits and adjustments will be needed. Common refinements include ensuring correct drawable imports and importing icons. Consider the generated code a highly efficient starting point, accelerating your UI development workflow.

Diagram explanation and documentation

With Gemini's multimodal capabilities, you can also try uploading an image of your diagram and ask for explanations or documentation.

Example prompt: Upload the Now in Android architecture diagram and say "Explain the components and data flow in this diagram" or “Write documentation about this diagram”.

Example of generating Compose code from high-fidelity mock using Gemini in Android Studio
4. Example of asking Gemini to help document the NowInAndroid architecture diagram

UI troubleshooting

Leverage Gemini's visual analysis to identify and resolve bugs quickly. Upload a screenshot of the problematic UI, and Gemini will analyze the image and suggest potential solutions. You can also include relevant code snippets for more precise assistance.

In the example below, we used Compose UI check and found that the button is stretched too wide in tablet screens, so we took a screenshot and asked Gemini for solutions - it was able to leverage the window size classes to provide the right fix.

Example of generating Compose code from high-fidelity mock using Gemini in Android Studio
5. Example of fixing UI bugs using Image Attachment (code output)

Download Android Studio today

Download the latest Android Studio canary today to try the new multimodal features!

As always, Google is committed to the responsible use of AI. Android Studio won't send any of your source code to servers without your consent. You can read more on Gemini in Android Studio's commitment to privacy.

We appreciate any feedback on things you like or features you would like to see. If you find a bug, please report the issue and also check out known issues. Remember to also follow us on X, Medium, or YouTube for more Android development updates!

Meet the Android Studio Team: A Conversation with Product Manager, Paris Hsu

Posted by Ashley Tschudin – Social Media Specialist, MTP at Google

Welcome to "Meet the Android Studio Team"; a short blog series where we pull back the curtain and introduce you to the passionate people who build your favorite Android development tools. Get to know the talented minds – engineers, designers, product managers, and more – who pour their hearts into crafting the best possible experience for Android developers.

Join us each week to meet a new member of the team and explore their unique perspectives.

Paris Hsu: Empowering Android developers with Compose tools

Meet Paris Hsu, a Product Manager at Google passionate about empowering developers to build incredible Android apps.

Her journey to the Android Studio team started with a serendipitous internship at Microsoft, where she discovered the power of developer tools. Now, as part of the UI Tools team, Paris champions intuitive solutions that streamline the development process, like the innovative Compose Tools suite.

In this installment of "Meet the Android Studio Team," Paris shares insights into her work, the importance of developer feedback, and her dream Android feature (hint: it involves acing that forehand).


Can you tell us about your journey to becoming a part of the Android Studio team? What sparked your interest in Android development?

Honestly, I joined a bit by chance! The summer before my last year of grad school, I was in the Microsoft's Garage incubator internship program. Our project, InkToCode, turned handwritten designs into code. It was my first experience building developer tools and made me realize how powerful developer tools can be, which led me to the Android Studio team. Now, after 6 years, I'm constantly amazed by what Android developers create – from innovative productivity apps to immersive games. It's incredibly rewarding to build tools that empower developers to create more.

In your opinion, what is the most impactful feature or improvement the Android Studio team has introduced in recent years, and why?

As part of the UI Tools team in Android Studio, I'm biased towards Compose Tools! Our team spent a lot of time rethinking how we can take a code-first approach for tools as we transition the community for XML to Compose. Features like the Compose Preview and its submodes (Interactive, Animation, Deploy preview) enable fast UI iteration, while features such as Layout Inspector or Compose UI Check helps find and diagnose UI issues with ease. We are also exploring ways to apply multimodal AI into these tools to help developers write more high quality, adaptive, and inclusive Compose code quicker.

How does the Android Studio team ensure that products or features meet the ever-changing needs of developers?

We are constantly engaging and listening to developer feedback to ensure we are meeting their needs! some examples:

    • Direct feedback: UXR studies, Annual developer surveys, and Buganizer reports provide valuable insights.
    • Early access: We release Early Access Programs (EAPs) for new features, allowing developers to test them and provide feedback before official launch.
    • Community engagement: We have advisory boards with experienced Android developers, gather feedback from Google Developer Experts (GDEs), and attend conferences to connect directly with the community.

How does the Studio team contribute to Google's broader vision for the Android platform?

I think Android Studio contributes to Google's broader mission by providing Android developers with powerful and intuitive tools. This way, developers are empowered to create amazing apps that bring the best of Google's services and information to our users. Whether it's accessing knowledge through Search, leveraging Gemini, staying connected with Maps, or enjoying entertainment on YouTube, Android Studio helps developers build the experiences that connect people to what matters most.

If you could wave a magic wand and add one dream feature to the Android universe, what would it be and why?

Anyone who knows me knows that I am recently super obsessed with tennis. I would love to see more coaching wearables (e.g. Pixel Watch, Pixel Racket?!). I would love real-time feedback on my serve and especially forehand stroke analysis.

Learn more about Compose Tools

Inspired by Paris’ passion for empowering developers to build incredible Android apps? To learn more about how Compose Tools can streamline your app development process, check out the Compose Tools documentation and get started with the Jetpack Compose Tutorial.

Stay tuned

Keep an eye out for the next installment in our “Meet the Android Studio Team” series, where we’ll shine the spotlight on another team member and delve into their unique insights.

Find Paris Hsu on LinkedIn, X, and Medium.