Unofficial Bevy Cheat Book

This is a reference-style book for the Bevy game engine (GitHub).

It aims to teach Bevy concepts in a concise way, help you be productive, and discover the knowledge you need.

This book aggregates a lot of community wisdom that is often not covered by official documentation, saving you the need to struggle with issues that others have figured out already!

While it aims to be exhaustive, documenting an entire game engine is a monumental task. I focus my time on whatever I believe the community needs most.

Therefore, there are still a lot of omissions, both for basics and advanced topics. Nevertheless, I am confident this book will prove to be a valuable resource to you!

Welcome! May this book serve you well!

(don't forget to Star the book's GitHub repository, and consider donating 🙂)

How to use this book

The pages in this book are not designed to be read in order. Each page covers a standalone topic. Feel free to jump to whatever interests you.

If you have a specific topic in mind that you would like to learn about, you can find it from the table-of-contents (sidebar) or using the search function (in the top bar).

The Chapter Overview page will give you a general idea of how the book is structured.

The text on each page will link to other pages, where you can learn about other things mentioned in the text. This helps you jump around the book.

If you are new to Bevy, or would like a more guided experience, try the Guided Tour tutorial. It will help you navigate the book in an order that makes sense for learning, from beginner to advanced topics.

The Bevy Builtins page is a concise cheatsheet of useful information about types and features provided by Bevy.

Bevy has a rich collection of official code examples.

Check out bevy-assets, for community-made resources.

Our community is very friendly and helpful. Feel welcome to join the Bevy Discord to chat, ask questions, or get involved in the project!

If you want to see some games made with Bevy, see itch.io or Bevy Assets.

Is this book up to date?

Bevy has a very rapid pace of development, with new major releases roughly every three months. Every version brings a lot of changes, so keeping this book updated can be a major challenge.

To ease the maintenance burden, the policy of the project is that the book may contain content for different versions of Bevy. However, mixing Bevy versions on the same page is not allowed.

At the top of every page, you will see the version it was last updated for. All content on that page must be relevant for the stated Bevy version.

Support Me

If you like this book, please consider sponsoring me. Thank you! ❤️

I'd like to keep improving and maintaining this book, to provide a high-quality independent learning resource for the Bevy community.

Support Bevy

If you like the Bevy Game Engine, you should consider donating to the project.

License

Copyright © 2021-2024 Ida Borisova (IyesGames)

All code in the book is provided under the MIT-0 License. At your option, you may also use it under the regular MIT License.

The text of the book is provided under the CC BY-NC-SA 4.0.

Exception: If used for the purpose of contribution to the "Official Bevy Project", the entire content of the book may be used under the MIT-0 License.

"Official Bevy Project" is defined as:

The MIT-0 license applies as soon as your contribution has been accepted upstream.

GitHub Forks and Pull Requests created for the purposes of contributing to the Official Bevy Project are given the following license exception: the Attribution requirements of CC BY-NC-SA 4.0 are waived for as long as the work is pending upstream review (Pull Request Open). If upstream rejects your contribution, you are given a period of 1 month to comply with the full terms of the CC BY-NC-SA 4.0 license or delete your work. If upstream accepts your contribution, the MIT-0 license applies.

Contributions

Development of this book is hosted on GitHub.

Please file GitHub Issues for any wrong/confusing/misleading information, as well as suggestions for new content you'd like to be added to the book.

Please do not create PRs. Or if you do, be prepared for them to be ignored or closed if I find that they take up too much of my time or don't help me enough.

See the Contributing section for all the details.

Stability Warning

Bevy is still a new and experimental game engine! It has only been public since August 2020!

While improvements have been happening at an incredible pace, and development is active, Bevy simply hasn't yet had the time to mature.

There are no stability guarantees and breaking changes happen often!

Usually, it not hard to adapt to changes with new releases, but you have been warned!

Bevy Version:(any)

Chapter Overview

The Bevy Builtins page is a concise cheatsheet of useful information about types and features provided by Bevy.

The Bevy Tutorials chapter is for tutorials/guides that you can follow from start to finish.

The Bevy Cookbook is for more self-contained / narrow-scoped examples that teach you how to solve specific problems.

The rest of the book is designed as a reference, covering different aspects of working with Bevy. Feel free to jump around the book, to learn about any topic that interests you. On every page of the book, any time other topics are mentioned, the relevant pages or official API documentation is linked.

If you would like a guided experience, or to browse the book by relative difficulty (from beginner to advanced), try the guided tutorial page. It recommends topics in a logical order for learning.

The book has the following general chapters:

To learn how to program in Bevy, see these chapters:

The following chapters cover various Bevy feature areas:

Bevy Version:0.11(outdated!)

As this page is outdated, please refer to Bevy's official migration guides while reading, to cover the differences: 0.11 to 0.12, 0.12 to 0.13, 0.13 to 0.14. 0.14 to 0.15.

I apologize for the inconvenience. I will update the page as soon as I find the time.

List of Bevy Builtins

This page is a quick condensed listing of all the important things provided by Bevy.

SystemParams

These are all the special types that can be used as system parameters.

(List in API Docs)

In regular systems:

In exclusive systems:

  • [&mut World]: Full direct access to the ECS World
  • [Local<T>]: Data local to the system
  • [&mut SystemState<P>][SystemState]: Emulates a regular system, allowing you to easily access data from the World. P are the system parameters.
  • [&mut QueryState<Q, F = ()>][QueryState]: Allows you to perform queries on the World, similar to a [Query] in regular systems.

Your function can have a maximum of 16 total parameters. If you need more, group them into tuples to work around the limit. Tuples can contain up to 16 members, but can be nested indefinitely.

Systems running during the Extract schedule can also use Extract<T>, to access data from the Main World instead of the Render World. T can be any read-only system parameter type.

Assets

(more info about working with assets)

These are the Asset types registered by Bevy by default.

  • Image: Pixel data, used as a texture for 2D and 3D rendering; also contains the SamplerDescriptor for texture filtering settings
  • TextureAtlas: 2D "Sprite Sheet" defining sub-images within a single larger image
  • Mesh: 3D Mesh (geometry data), contains vertex attributes (like position, UVs, normals)
  • Shader: GPU shader code, in one of the supported languages (WGSL/SPIR-V/GLSL)
  • ColorMaterial: Basic "2D material": contains color, optionally an image
  • StandardMaterial: "3D material" with support for Physically-Based Rendering
  • AnimationClip: Data for a single animation sequence, can be used with AnimationPlayer
  • Font: Font data used for text rendering
  • Scene: Scene composed of literal ECS entities to instantiate
  • DynamicScene: Scene composed with dynamic typing and reflection
  • Gltf: GLTF Master Asset: index of the entire contents of a GLTF file
  • GltfNode: Logical GLTF object in a scene
  • GltfMesh: Logical GLTF 3D model, consisting of multiple GltfPrimitives
  • GltfPrimitive: Single unit to be rendered, contains the Mesh and Material to use
  • AudioSource: Audio data for bevy_audio
  • FontAtlasSet: (internal use for text rendering)
  • SkinnedMeshInverseBindposes: (internal use for skeletal animation)

File Formats

These are the asset file formats (asset loaders) supported by Bevy. Support for each one can be enabled/disabled using cargo features. Some are enabled by default, many are not.

Image formats (loaded as Image assets):

FormatCargo featureDefault?Filename extensions
PNG"png"Yes.png
HDR"hdr"Yes.hdr
KTX2"ktx2"Yes.ktx2
KTX2+zstd"ktx2", "zstd"Yes.ktx2
JPEG"jpeg"No.jpg, .jpeg
WebP"webp"No.webp
OpenEXR"exr"No.exr
TGA"tga"No.tga
PNM"pnm"No.pam, .pbm, .pgm, .ppm
BMP"bmp"No.bmp
DDS"dds"No.dds
KTX2+zlib"ktx2", "zlib"No.ktx2
Basis"basis-universal"No.basis

Audio formats (loaded as AudioSource assets):

FormatCargo featureDefault?Filename extensions
OGG Vorbis"vorbis"Yes.ogg, .oga, .spx
FLAC"flac"No.flac
WAV"wav"No.wav
MP3"mp3"No.mp3

3D asset (model or scene) formats:

FormatCargo featureDefault?Filename extensions
GLTF"bevy_gltf"Yes.gltf, .glb

Shader formats (loaded as Shader assets):

FormatCargo featureDefault?Filename extensions
WGSLn/aYes.wgsl
GLSL"shader_format_glsl"No.vert, .frag, .comp
SPIR-V"shader_format_spirv"No.spv

Font formats (loaded as Font assets):

FormatCargo featureDefault?Filename extensions
TrueTypen/aYes.ttf
OpenTypen/aYes.otf

Bevy Scenes:

FormatFilename extensions
RON-serialized scene.scn,.scn.ron

There are unofficial plugins available for adding support for even more file formats.

GLTF Asset Labels

Asset path labels to refer to GLTF sub-assets.

The following asset labels are supported ({} is the numerical index):

  • Scene{}: GLTF Scene as Bevy Scene
  • Node{}: GLTF Node as GltfNode
  • Mesh{}: GLTF Mesh as GltfMesh
  • Mesh{}/Primitive{}: GLTF Primitive as Bevy Mesh
  • Mesh{}/Primitive{}/MorphTargets: Morph target animation data for a GLTF Primitive
  • Texture{}: GLTF Texture as Bevy Image
  • Material{}: GLTF Material as Bevy StandardMaterial
  • DefaultMaterial: as above, if the GLTF file contains a default material with no index
  • Animation{}: GLTF Animation as Bevy AnimationClip
  • Skin{}: GLTF mesh skin as Bevy SkinnedMeshInverseBindposes

Shader Imports

TODO

wgpu Backends

wgpu (and hence Bevy) supports the following backends:

PlatformBackends (in order of priority)
LinuxVulkan, GLES3
WindowsDirectX 12, Vulkan, GLES3
macOSMetal
iOSMetal
AndroidVulkan, GLES3
WebWebGPU, WebGL2

On GLES3 and WebGL2, some renderer features are unsupported and performance is worse.

WebGPU is experimental and few browsers support it.

Schedules

Internally, Bevy has these built-in schedules:

  • Main: runs every frame update cycle, to perform general app logic
  • ExtractSchedule: runs after Main, to copy data from the Main World into the Render World
  • Render: runs after ExtractSchedule, to perform all rendering/graphics, in parallel with the next Main run

The Main schedule simply runs a sequence of other schedules:

On the first run (first frame update of the app):

On every run (controlled via the MainScheduleOrder resource):

  • First: any initialization that must be done at the start of every frame
  • PreUpdate: for engine-internal systems intended to run before user logic
  • StateTransition: perform any pending state transitions
  • RunFixedUpdateLoop: runs the FixedUpdate schedule as many times as needed
  • Update: for all user logic (your systems) that should run every frame
  • PostUpdate: for engine-internal systems intended to run after user logic
  • Last: any final cleanup that must be done at the end of every frame

FixedUpdate is for all user logic (your systems) that should run at a fixed timestep.

StateTransition runs the OnEnter(...)/OnTransition(...)/OnExit(...) schedules for your states, when you want to change state.

The Render schedule is organized using sets (RenderSet):

  • ExtractCommands: apply deferred buffers from systems that ran in ExtractSchedule
  • Prepare/PrepareFlush: set up data on the GPU (buffers, textures, etc.)
  • Queue/QueueFlush: generate the render jobs to be run (usually phase items)
  • PhaseSort/PhaseSortFlush: sort and batch phase items for efficient rendering
  • Render/RenderFlush: execute the render graph to actually trigger the GPU to do work
  • Cleanup/CleanupFlush: clear any data from the render World that should not persist to the next frame

The *Flush variants are just to apply any deferred buffers after every step, if needed.

Run Conditions

TODO

Plugins

TODO

Bundles

Bevy's built-in bundle types, for spawning different common kinds of entities.

(List in API Docs)

Any tuples of up to 15 Component types are valid bundles.

General:

Scenes:

Audio:

Bevy 3D:

Bevy 2D:

Bevy UI:

Resources

(more info about working with resources)

Configuration Resources

These resources allow you to change the settings for how various parts of Bevy work.

These may be inserted at the start, but should also be fine to change at runtime (from a system):

  • ClearColor: Global renderer background color to clear the window at the start of each frame
  • GlobalVolume: The overall volume for playing audio
  • AmbientLight: Global renderer "fake lighting", so that shadows don't look too dark / black
  • Msaa: Global renderer setting for Multi-Sample Anti-Aliasing (some platforms might only support the values 1 and 4)
  • UiScale: Global scale value to make all UIs bigger/smaller
  • GizmoConfig: Controls how gizmos are rendered
  • WireframeConfig: Global toggle to make everything be rendered as wireframe
  • GamepadSettings: Gamepad input device settings, like joystick deadzones and button sensitivities
  • WinitSettings: Settings for the OS Windowing backend, including update loop / power-management settings
  • TimeUpdateStrategy: Used to control how the Time is updated
  • Schedules: Stores all schedules, letting you register additional functionality at runtime
  • MainScheduleOrder: The sequence of schedules that will run every frame update

Settings that are not modifiable at runtime are not represented using resources. Instead, they are configured via the respective plugins.

Engine Resources

These resources provide access to different features of the game engine at runtime.

Access them from your systems, if you need their state, or to control the respective parts of Bevy. These resources are in the Main World. See here for the resources in the Render World.

Render World Resources

These resources are present in the Render World. They can be accessed from rendering systems (that run during render stages).

  • MainWorld: (extract schedule only!) access data from the Main World
  • RenderGraph: The Bevy Render Graph
  • PipelineCache: Bevy's manager of render pipelines. Used to store render pipelines used by the app, to avoid recreating them more than once.
  • TextureCache: Bevy's manager of temporary textures. Useful when you need textures to use internally during rendering.
  • DrawFunctions<P>: Stores draw functions for a given phase item type
  • RenderAssets<T>: Contains handles to the GPU representations of currently loaded asset data
  • DefaultImageSampler: The default sampler for Image asset textures
  • FallbackImage: Dummy 1x1 pixel white texture. Useful for shaders that normally need a texture, when you don't have one available.

There are many other resources in the Render World, which are not mentioned here, either because they are internal to Bevy's rendering algorithms, or because they are just extracted copies of the equivalent resources in the Main World.

Low-Level wgpu Resources

Using these resources, you can have direct access to the wgpu APIs for controlling the GPU. These are available in both the Main World and the Render World.

  • RenderDevice: The GPU device, used for creating hardware resources for rendering/compute
  • [RenderQueue][bevy::RenderQueue]: The GPU queue for submitting work to the hardware
  • RenderAdapter: Handle to the physical GPU hardware
  • RenderAdapterInfo: Information about the GPU hardware that Bevy is running on

Input Handling Resources

These resources represent the current state of different input devices. Read them from your systems to handle user input.

Events

(more info about working with events)

Input Events

These events fire on activity with input devices. Read them to [handle user input][cb::input].

  • MouseButtonInput: Changes in the state of mouse buttons
  • MouseWheel: Scrolling by a number of pixels or lines (MouseScrollUnit)
  • MouseMotion: Relative movement of the mouse (pixels from previous frame), regardless of the OS pointer/cursor
  • CursorMoved: New position of the OS mouse pointer/cursor
  • KeyboardInput: Changes in the state of keyboard keys (keypresses, not text)
  • ReceivedCharacter: Unicode text input from the OS (correct handling of the user's language and layout)
  • Ime: Unicode text input from IME (support for advanced text input in different scripts)
  • TouchInput: Change in the state of a finger touching the touchscreen
  • GamepadEvent: Changes in the state of a gamepad or any of its buttons or axes
  • GamepadRumbleRequest: Send these events to control gamepad rumble
  • TouchpadMagnify: Pinch-to-zoom gesture on laptop touchpad (macOS)
  • TouchpadRotate: Two-finger rotate gesture on laptop touchpad (macOS)

Engine Events

Events related to various internal things happening during the normal runtime of a Bevy app.

System and Control Events

Events from the OS / windowing system, or to control Bevy.

Components

The complete list of individual component types is too specific to be useful to list here.

See: (List in API Docs)

Bevy Version:(any)

Bevy Tutorials

This chapter of the book contains tutorials. Tutorials teach you things in a logical order from start to finish. If you are looking for something to guide you through learning Bevy, maybe some of them will be useful to you.

The rest of this book is designed to be used as a reference, so you can jump around to specific topics you want to learn about.

The first tutorial in this chapter, Guided Tour, simply organizes all the topics in this book in an order suggested for learning, from the basics to advanced concepts. You can use it as an alternative to the main table of contents (the left side bar), if you are just learning Bevy and don't know how to progress. If you are new to Bevy, you can start here to find your way around.

If you would like more narrow-scoped examples that teach you how to solve specific problems, those can be found in the Bevy Cookbook chapter.

You should also look at Bevy's official collection of examples. There is something for almost every area of the engine, though they usually only show simple usage of the APIs without much explanation.

Bevy Version:0.13(outdated!)

As this page is outdated, please refer to Bevy's official migration guides while reading, to cover the differences: 0.13 to 0.14. 0.14 to 0.15.

I apologize for the inconvenience. I will update the page as soon as I find the time.

New to Bevy? Guided Tutorial!

Welcome to Bevy! :) We are glad to have you in our community!

This page will guide you through this book, to help you gain comprehensive knowledge of how to work with Bevy. The topics are structured in an order that makes sense for learning: from basics to advanced.

It is just a suggestion to help you navigate. Feel free to jump around the book and read whatever interests you. The main table-of-contents (the left sidebar) was designed to be a reference for Bevy users of any skill level.

Make sure to also look at the official Bevy examples. If you need help, use GitHub Discussions, or feel welcome to join us to chat and ask for help in Discord.

If you run into issues, be sure to check the Common Pitfalls chapter, to see if this book has something to help you. Solutions to some of the most common issues that Bevy community members have encountered are documented there.

Basics

These are the absolute essentials of using Bevy. Every Bevy project, even a simple one, would require you to be familiar with these concepts.

You could conceivably make something like a simple game-jam game or prototype, using just this knowledge. Though, as your project grows, you will likely quickly need to learn more.

Next Steps

You will likely need to learn most of these topics to make a non-trivial Bevy project. After you are confident with the basics, you should learn these.

Intermediate

These are more specialized topics. You may need some of them, depending on your project.

Advanced

These topics are for niche technical situations. You can learn them, if you want to know more about how Bevy works internally, extend the engine with custom functionality, or do other advanced things with Bevy.

Bevy Version:(any)

Bevy Cookbook

This chapter shows you how to do various practical things using Bevy.

Every page is focused on a specific problem and provides explanations and example code to teach you how to solve it.

It is assumed that you are already familiar with Bevy Programming.

You should also look at Bevy's official collection of examples. There is something for almost every area of the engine, though they usually only show simple usage of the APIs without much explanation.

If you would like step-by-step tutorials that you can follow from start to finish, those are in the Bevy Tutorials chapter.

Bevy Version:0.13(outdated!)

As this page is outdated, please refer to Bevy's official migration guides while reading, to cover the differences: 0.13 to 0.14. 0.14 to 0.15.

I apologize for the inconvenience. I will update the page as soon as I find the time.

Show Framerate

You can use Bevy's builtin diagnostics to measure framerate (FPS), for monitoring performance.

To enable it, add Bevy's diagnostic plugin to your app:

use bevy::diagnostic::FrameTimeDiagnosticsPlugin;
app.add_plugins(FrameTimeDiagnosticsPlugin::default());

Print to Console / Log

The simplest way to use it is to print the diagnostics to the console (log). If you want to only do it in dev builds, you can add a conditional-compilation attribute.

#[cfg(debug_assertions)] // debug/dev builds only
{
    use bevy::diagnostic::LogDiagnosticsPlugin;
    app.add_plugins(LogDiagnosticsPlugin::default());
}

In-Game / On-Screen FPS counter

UPDATE! I have now released a Bevy plugin which provides a much better version of the code on this page, ready for you to use! Consider trying my iyes_perf_ui plugin!

Bevy maintainers have expressed interest in upstreaming it, and we will try to make it official in the next Bevy release (0.14)!

For now, I am also keeping the old code example below in the book, for completeness:

You can use Bevy UI to create an in-game FPS counter.

It is recommended that you create a new UI root (entity without a parent) with absolute positioning, so that you can control the exact position where the FPS counter appears, and so it doesn't affect the rest of your UI.

Here is some example code showing you how to make a very nice-looking and readable FPS counter:

Code Example (Long): 
use bevy::diagnostic::DiagnosticsStore;
use bevy::diagnostic::FrameTimeDiagnosticsPlugin;

/// Marker to find the container entity so we can show/hide the FPS counter
#[derive(Component)]
struct FpsRoot;

/// Marker to find the text entity so we can update it
#[derive(Component)]
struct FpsText;

fn setup_fps_counter(
    mut commands: Commands,
) {
    // create our UI root node
    // this is the wrapper/container for the text
    let root = commands.spawn((
        FpsRoot,
        NodeBundle {
            // give it a dark background for readability
            background_color: BackgroundColor(Color::BLACK.with_a(0.5)),
            // make it "always on top" by setting the Z index to maximum
            // we want it to be displayed over all other UI
            z_index: ZIndex::Global(i32::MAX),
            style: Style {
                position_type: PositionType::Absolute,
                // position it at the top-right corner
                // 1% away from the top window edge
                right: Val::Percent(1.),
                top: Val::Percent(1.),
                // set bottom/left to Auto, so it can be
                // automatically sized depending on the text
                bottom: Val::Auto,
                left: Val::Auto,
                // give it some padding for readability
                padding: UiRect::all(Val::Px(4.0)),
                ..Default::default()
            },
            ..Default::default()
        },
    )).id();
    // create our text
    let text_fps = commands.spawn((
        FpsText,
        TextBundle {
            // use two sections, so it is easy to update just the number
            text: Text::from_sections([
                TextSection {
                    value: "FPS: ".into(),
                    style: TextStyle {
                        font_size: 16.0,
                        color: Color::WHITE,
                        // if you want to use your game's font asset,
                        // uncomment this and provide the handle:
                        // font: my_font_handle
                        ..default()
                    }
                },
                TextSection {
                    value: " N/A".into(),
                    style: TextStyle {
                        font_size: 16.0,
                        color: Color::WHITE,
                        // if you want to use your game's font asset,
                        // uncomment this and provide the handle:
                        // font: my_font_handle
                        ..default()
                    }
                },
            ]),
            ..Default::default()
        },
    )).id();
    commands.entity(root).push_children(&[text_fps]);
}

fn fps_text_update_system(
    diagnostics: Res<DiagnosticsStore>,
    mut query: Query<&mut Text, With<FpsText>>,
) {
    for mut text in &mut query {
        // try to get a "smoothed" FPS value from Bevy
        if let Some(value) = diagnostics
            .get(&FrameTimeDiagnosticsPlugin::FPS)
            .and_then(|fps| fps.smoothed())
        {
            // Format the number as to leave space for 4 digits, just in case,
            // right-aligned and rounded. This helps readability when the
            // number changes rapidly.
            text.sections[1].value = format!("{value:>4.0}");

            // Let's make it extra fancy by changing the color of the
            // text according to the FPS value:
            text.sections[1].style.color = if value >= 120.0 {
                // Above 120 FPS, use green color
                Color::rgb(0.0, 1.0, 0.0)
            } else if value >= 60.0 {
                // Between 60-120 FPS, gradually transition from yellow to green
                Color::rgb(
                    (1.0 - (value - 60.0) / (120.0 - 60.0)) as f32,
                    1.0,
                    0.0,
                )
            } else if value >= 30.0 {
                // Between 30-60 FPS, gradually transition from red to yellow
                Color::rgb(
                    1.0,
                    ((value - 30.0) / (60.0 - 30.0)) as f32,
                    0.0,
                )
            } else {
                // Below 30 FPS, use red color
                Color::rgb(1.0, 0.0, 0.0)
            }
        } else {
            // display "N/A" if we can't get a FPS measurement
            // add an extra space to preserve alignment
            text.sections[1].value = " N/A".into();
            text.sections[1].style.color = Color::WHITE;
        }
    }
}

/// Toggle the FPS counter when pressing F12
fn fps_counter_showhide(
    mut q: Query<&mut Visibility, With<FpsRoot>>,
    kbd: Res<ButtonInput<KeyCode>>,
) {
    if kbd.just_pressed(KeyCode::F12) {
        let mut vis = q.single_mut();
        *vis = match *vis {
            Visibility::Hidden => Visibility::Visible,
            _ => Visibility::Hidden,
        };
    }
}
app.add_systems(Startup, setup_fps_counter);
app.add_systems(Update, (
    fps_text_update_system,
    fps_counter_showhide,
));
Bevy Version:0.13(outdated!)

As this page is outdated, please refer to Bevy's official migration guides while reading, to cover the differences: 0.13 to 0.14. 0.14 to 0.15.

I apologize for the inconvenience. I will update the page as soon as I find the time.

Convert cursor to world coordinates

2D games

If you only have one window (the primary window), as is the case for most apps and games, you can do this:

Code (simple version): 
use bevy::window::PrimaryWindow;

/// We will store the world position of the mouse cursor here.
#[derive(Resource, Default)]
struct MyWorldCoords(Vec2);

/// Used to help identify our main camera
#[derive(Component)]
struct MainCamera;

fn setup(mut commands: Commands) {
    // Make sure to add the marker component when you set up your camera
    commands.spawn((Camera2dBundle::default(), MainCamera));
}

fn my_cursor_system(
    mut mycoords: ResMut<MyWorldCoords>,
    // query to get the window (so we can read the current cursor position)
    q_window: Query<&Window, With<PrimaryWindow>>,
    // query to get camera transform
    q_camera: Query<(&Camera, &GlobalTransform), With<MainCamera>>,
) {
    // get the camera info and transform
    // assuming there is exactly one main camera entity, so Query::single() is OK
    let (camera, camera_transform) = q_camera.single();

    // There is only one primary window, so we can similarly get it from the query:
    let window = q_window.single();

    // check if the cursor is inside the window and get its position
    // then, ask bevy to convert into world coordinates, and truncate to discard Z
    if let Some(world_position) = window.cursor_position()
        .and_then(|cursor| camera.viewport_to_world(camera_transform, cursor))
        .map(|ray| ray.origin.truncate())
    {
        mycoords.0 = world_position;
        eprintln!("World coords: {}/{}", world_position.x, world_position.y);
    }
}
app.init_resource::<MyWorldCoords>();
app.add_systems(Startup, setup);
app.add_systems(Update, my_cursor_system);

If you have a more complex application with multiple windows, here is a more complex version of the code that can handle that:

Code (multi-window version): 
use bevy::render::camera::RenderTarget;
use bevy::window::WindowRef;

/// We will add this to each camera we want to compute cursor position for.
/// Add the component to the camera that renders to each window.
#[derive(Component, Default)]
struct WorldCursorCoords(Vec2);

fn setup_multiwindow(mut commands: Commands) {
    // TODO: set up multiple cameras for multiple windows.
    // See bevy's example code for how to do that.

    // Make sure we add our component to each camera
    commands.spawn((Camera2dBundle::default(), WorldCursorCoords::default()));
}

fn my_cursor_system_multiwindow(
    // query to get the primary window
    q_window_primary: Query<&Window, With<PrimaryWindow>>,
    // query to get other windows
    q_window: Query<&Window>,
    // query to get camera transform
    mut q_camera: Query<(&Camera, &GlobalTransform, &mut WorldCursorCoords)>,
) {
    for (camera, camera_transform, mut worldcursor) in &mut q_camera {
        // get the window the camera is rendering to
        let window = match camera.target {
            // the camera is rendering to the primary window
            RenderTarget::Window(WindowRef::Primary) => {
                q_window_primary.single()
            },
            // the camera is rendering to some other window
            RenderTarget::Window(WindowRef::Entity(e_window)) => {
                q_window.get(e_window).unwrap()
            },
            // the camera is rendering to something else (like a texture), not a window
            _ => {
                // skip this camera
                continue;
            }
        };

        // check if the cursor is inside the window and get its position
        // then, ask bevy to convert into world coordinates, and truncate to discard Z
        if let Some(world_position) = window.cursor_position()
            .and_then(|cursor| camera.viewport_to_world(camera_transform, cursor))
            .map(|ray| ray.origin.truncate())
        {
            worldcursor.0 = world_position;
        }
    }
}
app.add_systems(Startup, setup_multiwindow);
app.add_systems(Update, my_cursor_system_multiwindow);

3D games

If you'd like to be able to detect what 3D object the cursor is pointing at, select objects, etc., there is a good (unofficial) plugin: bevy_mod_picking.

For a simple top-down camera view game with a flat ground plane, it might be sufficient to just compute the coordinates on the ground under the cursor.

In the interactive example, there is a ground plane with a non-default position and rotation. There is a red cube, which is positioned using the global coordinates, and a blue cube, which is a child entity of the ground plane and positioned using local coordinates. They should both follow the cursor.

Code and explanation: 
/// Here we will store the position of the mouse cursor on the 3D ground plane.
#[derive(Resource, Default)]
struct MyGroundCoords {
    // Global (world-space) coordinates
    global: Vec3,
    // Local (relative to the ground plane) coordinates
    local: Vec2,
}

/// Used to help identify our main camera
#[derive(Component)]
struct MyGameCamera;

/// Used to help identify our ground plane
#[derive(Component)]
struct MyGroundPlane;

fn setup_3d_scene(mut commands: Commands) {
    // Make sure to add the marker component when you set up your camera
    commands.spawn((
        MyGameCamera,
        Camera3dBundle {
            // ... your camera configuration ...
            ..default()
        },
    ));
    // Spawn the ground
    commands.spawn((
        MyGroundPlane,
        PbrBundle {
            // feel free to change this to rotate/tilt or reposition the ground
            transform: Transform::default(),
            // TODO: set up your mesh / visuals for rendering:
            // mesh: ...
            // material: ...
            ..default()
        },
    ));
}

fn cursor_to_ground_plane(
    mut mycoords: ResMut<MyGroundCoords>,
    // query to get the window (so we can read the current cursor position)
    // (we will only work with the primary window)
    q_window: Query<&Window, With<PrimaryWindow>>,
    // query to get camera transform
    q_camera: Query<(&Camera, &GlobalTransform), With<MyGameCamera>>,
    // query to get ground plane's transform
    q_plane: Query<&GlobalTransform, With<MyGroundPlane>>,
) {
    // get the camera info and transform
    // assuming there is exactly one main camera entity, so Query::single() is OK
    let (camera, camera_transform) = q_camera.single();

    // Ditto for the ground plane's transform
    let ground_transform = q_plane.single();

    // There is only one primary window, so we can similarly get it from the query:
    let window = q_window.single();

    // check if the cursor is inside the window and get its position
    let Some(cursor_position) = window.cursor_position() else {
        // if the cursor is not inside the window, we can't do anything
        return;
    };

    // Mathematically, we can represent the ground as an infinite flat plane.
    // To do that, we need a point (to position the plane) and a normal vector
    // (the "up" direction, perpendicular to the ground plane).

    // We can get the correct values from the ground entity's GlobalTransform
    let plane_origin = ground_transform.translation();
    let plane = Plane3d::new(ground_transform.up());

    // Ask Bevy to give us a ray pointing from the viewport (screen) into the world
    let Some(ray) = camera.viewport_to_world(camera_transform, cursor_position) else {
        // if it was impossible to compute for whatever reason; we can't do anything
        return;
    };

    // do a ray-plane intersection test, giving us the distance to the ground
    let Some(distance) = ray.intersect_plane(plane_origin, plane) else {
        // If the ray does not intersect the ground
        // (the camera is not looking towards the ground), we can't do anything
        return;
    };

    // use the distance to compute the actual point on the ground in world-space
    let global_cursor = ray.get_point(distance);

    mycoords.global = global_cursor;
    eprintln!("Global cursor coords: {}/{}/{}",
        global_cursor.x, global_cursor.y, global_cursor.z
    );

    // to compute the local coordinates, we need the inverse of the plane's transform
    let inverse_transform_matrix = ground_transform.compute_matrix().inverse();
    let local_cursor = inverse_transform_matrix.transform_point3(global_cursor);

    // we can discard the Y coordinate, because it should always be zero
    // (our point is supposed to be on the plane)
    mycoords.local = local_cursor.xz();
    eprintln!("Local cursor coords: {}/{}", local_cursor.x, local_cursor.z);
}
app.init_resource::<MyGroundCoords>();
app.add_systems(Startup, setup_3d_scene);
app.add_systems(Update, cursor_to_ground_plane);

If the ground is tilted/rotated or moved, the global and local coordinates will differ, and may be useful for different use cases, so we compute both.

For some examples:

  • if you want to spawn a child entity, or to quantize the coordinates to a grid (for a tile-based game, to detect the grid tile under the cursor), the local coordinates will be more useful
  • if you want to spawn some overlays, particle effects, other independent game entities, at the position of the cursor, the global coordinates will be more useful