Prevent resetting graph when slider plotly barplot R

Question

I would like to prevent plotly rerender the graph when we change the x-axis range by dates and when you deselect bars from the legend it should remain the same bars when it changed the x-axis. For this we could probably use plotlyProxyInvoke. According to the documentation the restyle option might work. The problem is when we deselect bars (for example from 4 to 2 bars) and we change the axis range by slider, it rerenders and brings all bars back. Here I tried to create a simple reproducible example:

library(shiny)
library(plotly)
library(dplyr)
library(shinyWidgets)

ui <- fluidPage(
  shinyWidgets::sliderTextInput(
    "date_range",
    "Select Date Range:",
    choices = seq(as.Date("2025-01-01"), as.Date("2025-12-31"), by = "month"),
    selected = c(as.Date("2025-01-01"), as.Date("2025-06-01"))
  ),
  plotlyOutput("barplot")
)

server <- function(input, output, session) {
  
  sample_data <- reactive({
    dates <- seq(as.Date("2025-01-01"), as.Date("2025-12-31"), by = "month")
    categories <- c("A", "B", "C", "D")
    
    expand.grid(date = dates, category = categories) %>%
      mutate(value = runif(n(), 10, 100))
  })
  
  filtered_data <- reactive({
    req(input$date_range)
    sample_data() %>%
      filter(date >= input$date_range[1], date <= input$date_range[2])
  })
  
  output$barplot <- renderPlotly({
    data <- filtered_data()
    plot_ly(data, x = ~date, y = ~value, color = ~category, type = "bar") %>%
      layout(barmode = "group")
  })
  
  observeEvent(input$date_range, {
    data <- filtered_data()
    plotly_proxy <- plotlyProxy("barplot", session)
    
    plotlyProxyInvoke(
      plotly_proxy,
      "restyle",
      list(
        x = list(data$date),
        y = list(data$value)
      )
    )
  })
}

shinyApp(ui, server)

Output:

This is fine, but when we for example first only want bar A and D, and then extend the x-axis range with slider. All bars still appear. So I was wondering if anyone knows how we can keep the bars when we filter the x-axis without rerendering the full plotly?