Convert cursor to world coordinates

Click here for the full example code.

3D games

There is a good (unofficial) plugin: bevy_mod_picking.

2D games

Starting from Bevy 0.9, there are camera methods to help you convert between screen-space (viewport) and world-space coordinates. We can use this to easily find the position of the mouse cursor.

This code will work regardless of the camera's projection settings and transform.

(there will likely be slight inaccuracy from floating-point calculations)

/// Used to help identify our main camera
#[derive(Component)]
struct MainCamera;

fn setup(mut commands: Commands) {
    commands.spawn((Camera2dBundle::default(), MainCamera));
}

fn my_cursor_system(
    // need to get window dimensions
    windows: Res<Windows>,
    // query to get camera transform
    camera_q: Query<(&Camera, &GlobalTransform), With<MainCamera>>,
) {
    // get the camera info and transform
    // assuming there is exactly one main camera entity, so query::single() is OK
    let (camera, camera_transform) = camera_q.single();

    // get the window that the camera is displaying to (or the primary window)
    let window = if let RenderTarget::Window(id) = camera.target {
        windows.get(id).unwrap()
    } else {
        windows.get_primary().unwrap()
    };

    // check if the cursor is inside the window and get its position
    // then, ask bevy to convert into world coordinates, and truncate to discard Z
    if let Some(world_position) = window.cursor_position()
        .and_then(|cursor| camera.viewport_to_world(camera_transform, cursor))
        .map(|ray| ray.origin.truncate())
    {
        eprintln!("World coords: {}/{}", world_position.x, world_position.y);
    }
}

In older versions of Bevy, there were no such coordinate conversion methods, and the math had to be done manually. If you are interested in how that works, see the example from an old version of the book, here.