This is the desired layout:

I can achieve a fixed aspect ratio with a 16:9 image and setting the img width 100%; However this way I lose the responsive height scaling of flexbox, the height remains the same and only the row1 resizes its height when resizing the window. The row 2 just overflows and doesn’t scale the height whatsoever. To solve this I tried to introduce a 3rd div in the middle to fill the remaining space for when the viewport height decreases, which would make the image1 and image2 smaller and the middle div wider.

My question is: Is it even possible to achieve this with flexbox or will I need to do it using css grid or js?

html:

<div id="container">
      <div id="row1">
            <div id="panel1"></div>
            <div id="panel2"></div>
      </div>
      <div id="row2">
            <div id="panel3">
              <img/>
            </div>
            <div id="panel4"></div>
            <div id="panel5">
               <img/>
       </div>
</div>

css:

#container {
    height: 100vh;
    width: 100vw;
    display: flex;
    flex-wrap: wrap;
    flex-direction: column;
}

#row1 {
    display: flex;
    flex-wrap: wrap;
    flex: 2;
    gap: 7px;
    width: 100%;
    padding: 7px;
}
#row2 {
    display: flex;
    flex-flow: row nowrap;
    flex: 1;
    gap: 7px;
    width: 100%;
    padding: 0 7px 7px 7px;
}

#panel1 {
    background: white;
    flex: 1;
}

#panel2 {
    background: white;
    width: 100px;
}

#panel3 {
    background: white;
    flex: 1;
}

#panel4 {
    background: white;
    flex: 1;
}

#panel5 {
    background: white;
    flex: 1;
}

#image1 {
width: 100%;
}

>Solution :

Remove the flex: 1 styling from panels 3 and 5 and set an aspect-ratio of 16/9.

#panel3 {
  aspect-ratio: 16 / 9;
}

#panel4 {
  flex: 1;
}

#panel5 {
  aspect-ratio: 16 / 9;
}

*, *::before, *::after {
  box-sizing: border-box;
}

html, body {
  width: 100%:
  height: 100%;
  margin: 0;
  padding: 0;
}

#container {
  height: 100vh;
  width: 100vw;
  display: flex;
  flex-wrap: wrap;
  flex-direction: column;
}

#row1 {
  display: flex;
  flex-wrap: wrap;
  flex: 2;
  gap: 7px;
  width: 100%;
  padding: 7px;
}

#row2 {
  display: flex;
  flex-flow: row nowrap;
  flex: 1;
  gap: 7px;
  width: 100%;
  padding: 0 7px 7px 7px;
}

#panel1 {
  background: red;
  flex: 1;
}

#panel2 {
  background: orange;
  width: 100px;
}

#panel3 {
  background: yellow;
  aspect-ratio: 16 / 9;
}

#panel4 {
  background: green;
  flex: 1;
}

#panel5 {
  background: blue;
  aspect-ratio: 16 / 9;
}

#image1 {
  width: 100%;
}

<div id="container">
  <div id="row1">
    <div id="panel1"></div>
    <div id="panel2"></div>
  </div>
  <div id="row2">
    <div id="panel3">
      <img/>
    </div>
    <div id="panel4"></div>
    <div id="panel5">
      <img/>
    </div>
  </div>
</div>