Hi, IF the IRF stack (which is a single logical entity from the standpoint of each Firewall appliance) is going to see - let me use this figurative terminology - both the Firewall appliances as a single logical entity (so the Firewalls Cluster is exactly like the IRF Stack and I don't believe this is what the IRF will see) then "Yes, you could create just two BAGGs, one per side" BUT if the answer is "No, the IRF will not see a single logical entity but two separate ones (Active and Passive appliances)" THEN "No, you can't setup just two BAGGs, one per side" and you can't also proceed with the proposed design because BAGGs sourcing from the IRF must co-terminate into a logical entity (a single Firewall Appliance).
You can create a BAGG sourcing from the Active Firewall appliance with its member links terminating into IRF Member 1 and IRF Member 2 and, concurrently, you can also create the another one BAGG sourcing from the Passive Firewall appliance with its member links terminating into IRF Member 1 and IRF Member 2 (basically links sourcing from each Firewall appliance are distributed into both IRF Members and are sourced from each Firewall appliance, not from both Firewall appliances as in the case of real Multi-Chassis LAGs), on IRF side you are going to create two corresponding BAGGs (one terminating its links into the Active Firewall appliance and the other one into the Passive Firewall appliance).
As written, the only exception shows up IF the Firewalls Cluster is going to act as a single logical entity from the upstream/downstream peers standpoint (from the IRF standpoint, in your case); generally, the presence of a feature like the Multi-Chassis LAG (MC-LAG) capability on a Stack shows that you're dealing with a Cluster which is capable of acting as a single logical entity and so BAGGs (MC-LAGs) can use links sourced by any of its members (but the termination should also be a single logical entity, the IRF in this case). Verify that.