Using Virtual Routing or Policy Based Rules to forward traffic to a secondary service provider (base scenario).

Security Gateway Articles and How to's
Post Reply
Peter
Posts: 617
Joined: 10 Apr 2008, 14:14
Location: Clavister HQ - Örnsköldsvik

Using Virtual Routing or Policy Based Rules to forward traffic to a secondary service provider (base scenario).

Post by Peter » 16 Nov 2018, 13:01

This How-to applies to:
  • Any Clavister Next Generation Firewall Core version but article is based on version 12.00
1. Objectives with this article:
The objective of this article is to describe how the same scenario can be solved using two different methods. One using Virtual Routing (VR) and the other using Policy Based Routing rules (PBR).

The Clavister cOS Core is very powerful when it comes to routing and there is almost always more than one way to solve a particular scenario.

2. Scenario:
The scenario we are going to describe is the classic / basic scenario involving VR or PBR. For people not used to using either of these features this article with hopefully give the reader a good start in understanding how these features can be used and to have a base scenario / configuration to build upon for future use.

The scenario is that we have one Firewall but two Internet Service Providers (ISP). We have two internal networks were we want each internal network to use their own ISP. Basically they should not share their Internet connection. This is further visualized in the below network schematic:

Important: This How-To focuses mainly on outgoing traffic, meaning traffic initiated from behind the Firewall to a server/host on the Internet in order to keep the scenario as simple as possible.

PBR_VR_Basic.png
PBR_VR_Basic.png (86.97 KiB) Viewed 335 times

If we have a look at how the routing and rules look before we add our secondary ISP it will look something like this:

2.1 Routing:

Flags  Network            Iface          Gateway         Local IP        Metric
------ ------------------ -------------- --------------- --------------- ------
       192.168.98.0/24    G1                                             100
       10.10.10.0/24      GESW                                           100
       192.168.10.0/24    Vlan_10                                        100
       172.16.10.0/24     IKEv2_Tunnel                                   100
       0.0.0.0/0          G1             192.168.98.1                    100

2.2 Rules:

001.png
001.png (45.39 KiB) Viewed 338 times

So as we can see both the routing and rules are currently configured to use the primary ISP on the G1 interface. We now want to add a secondary ISP behind the G2 interface.

Note: Please note that these examples are based on an internal lab setup. That is the reason why we are using private IP addresses on our external interfaces. In a real live environment this will of course not be the case.

3. Requirements:
We have three requirements that we need to fulfill in this scenario.
1. The users behind the GESW interface should use ISP-1 to reach the internet.
2. The users behind the Vlan_10 interface should use ISP-2 to reach the internet.
3. GESW and Vlan_10 should not be aware of each other or be able to communicate.

4. Solving the scenario using Virtual Routing (VR).
Once we have connected the G2 interface to the secondary ISP (ISP-2) and configured the G2 interface with the required IP addresses and networks it is now time to create the secondary routing table.

4.1. Creating the secondary routing table.

002.png
002.png (16.91 KiB) Viewed 338 times

The ordering of the routing table here is quite important but it can also come down to personal preference. There are three choices regarding Ordering:

Default
The default behavior is to first look up the route in the main table. If no matching route is
found, or a default route is found (a route with the destination all-nets), a lookup for a
matching route in the alternate table is performed. If no match is found in the alternate
table then the default route in the main table will be used.

First
This behavior is to first look up the connection's route in the alternate table. If no matching
route is found there then the main table is used for the lookup. The default all-nets route
will be counted as a match in the alternate table if it is found there.

Only
This option ignores the existence of any other table except the alternate table so that is the
only one used for the lookup.
One application of this option is to give the administrator a way to dedicate a single routing
table to one set of interfaces. The Only option should be used when creating virtual systems
since it can dedicate a routing table to a set of interfaces.

These options can be quite confusing, which one to use depends on the person configuring the system as we mentioned earlier there are usually several ways to accomplish the same thing and which method to use can boil down to personal preference.

But one method that could be considered easier to understand is if we use the option “Only”. Basically the principle for an “Only” routing table is that any packet that ends up in this routing table, stays in this routing table. So you need to make sure that all needed routes exist in this routing table if a packet ends up in this routing table.

Scenarios that want to lock down interfaces using VR and to avoid intercommunication between internal networks the Only option is required. For instance if we have a scenario where the same network is used behind multiple interfaces. VR and “Only” need to be used to avoid routing problems and communication spillover between similar networks. Basically it is a way to build waterproof bulkheads between the various internal networks.

4.2. Interface routing table memberships
Since the requirement is that G2 and Vlan_10 should be paired we move over to make these two interfaces a specific member of the routing table ISP-2 as shown in the below picture for the G2 interface.

003.png
003.png (45.61 KiB) Viewed 338 times

We repeat the same operation for the Vlan_10 interface. Once this is done we need to make sure that the correct routes are in this routing table. Since G2 should be configured similar to G1 in terms of network and gateway it should look something like this in the ISP-2 routing table:

Routing table: ISP_2

Flags  Network            Iface          Gateway         Local IP        Metric
------ ------------------ -------------- --------------- --------------- ------
       10.10.20.0/24      G2                                             100
       192.168.10.0/24    Vlan_10                                        100
       0.0.0.0/0          G2             10.10.20.1                      100

As we can see there is no references to G1 or the GESW interface at all in this routing table. And since both G2 and Vlan_10 is a specific member of this routing table it means that any other traffic that arrives in this routing table will be dropped by the Firewall (it should reasonably not happen though if all rules and routes are configured correctly).

4.3. Modifying the IP policy rules to use the secondary ISP.
The last thing we need to change is the IP policy rules. Since they are currently pointing towards the G1 interface in order to reach the internet we need to change them to point towards the G2 interface instead as that is the primary ISP to be used for users behind the Vlan_10 interface.

004.png
004.png (46.1 KiB) Viewed 338 times

That is what is needed for this scenario using VR and interface memberships. If a packet is sent from a client behind Vlan_10 the Firewall will pick up the packet and notice that the interface is part of the ISP-2 routing table. The Firewall will then look in the ISP-2 routing table to verify that the source interface is Vlan_10 and then try to find a matching route for the destination network.

4.4. Testing the rules and routes using ping simulations.
If we as an example try to ping a host on the Internet using a ping simulation, it will look like this:

E5:/> ping 8.8.8.8 -srcif=Vlan_10 -srcip=192.168.10.50 -verbose
Rule and routing information for ping:
PBR selected by rule "iface_member_ISP_2" - PBR table "ISP_2"
     allowed by rule "Vlan_10_Ping"

Sending 1 4-byte ICMP ping to 8.8.8.8 from 10.10.20.2
 sent via route "0.0.0.0/0 via G2, gw 10.10.20.1" in PBR table "ISP_2"

Ping Results:  Sent: 1, Received:1, Avg RTT: 10.0 ms

As we can see it correctly uses the secondary ISP interface (G2 interface) to reach the Internet.

And a last check to make sure that Vlan_10 is unable to reach the network behind GESW we run a second ping simulation to verify this:

E5:/> ping 10.10.10.25 -srcif=Vlan_10 -srcip=192.168.10.50 -verbose
Rule and routing information for ping:
PBR selected by rule "iface_member_ISP_2" - PBR table "ISP_2"
     allowed by rule "Vlan_10_Ping"

Sending 1 4-byte ICMP ping to 10.10.10.25 from 10.10.20.2
 sent via route "0.0.0.0/0 via G2, gw 10.10.20.1" in PBR table "ISP_2"

Ping Results:  Sent: 1, Received:0, Loss: 100%

Some may initially think that the above simulation output looks strange as it seems to allow the connection attempt, but it is correct. The Firewall is attempting to find the host with IP 10.10.10.25 and performs a route lookup for this destination IP in the ISP-2 routing table, and the matching route will be the All-Nets route towards the Internet. We have an IP policy that allows pings towards the internet so it will allow it and forward the request to the ISP-2 gateway.

The ping will of course fail as 10.10.10.25 is a private IP address that the router will most likely drop.

All requirements has been fulfilled using VR.

5. Solving the scenario using Policy Based Routing (PBR).

Before we start we will again go back to using the base routing and rule setup as shown in section 2.1. and 2.2. Once we have connected the G2 interface to the secondary ISP (ISP-2) and configured the G2 interface with the required IP addresses and networks it is now time to create the secondary routing table.

5.1. Creating the secondary routing table.
We will be using the exactly same routing ordering as when solving the scenario using VR.

002.png
002.png (16.91 KiB) Viewed 338 times
For information and details about the different route orderings please see section 4.1.

We will again be using ordering “Only” on our new routing table. The reason why we want to use “Only” is mainly due to personal preference but it is also easier to understand and create the PBR rules as by using “Only” one of the basic principles is that when something arrives in this routing table, it stays in this routing table.

This can be very useful as we then know that both the source and destination IP addresses must exist in this routing table in order for traffic to work correctly.

Since we are not using VR we do not change any settings on the interfaces, all interfaces will be a member of all routing tables (default setting). But we will need PBR rules in order to make traffic use this routing table instead of the default <Main> routing table for internet access.

5.2. Creating the needed routes in the secondary routing table.
The routes will be the same as when we solved it using the VR method and it will look like this in <Main> and our <ISP_2> routing table when we are done:


Routing table <Main>
E5:/> routes
Flags  Network            Iface          Gateway         Local IP        Metric
------ ------------------ -------------- --------------- --------------- ------
       192.168.98.0/24    G1                                             100
       10.10.10.0/24      GESW                                           100
       192.168.10.0/24    Vlan_10                                        100
       0.0.0.0/0          G1             192.168.98.1                    100
Routing table <ISP_2>

E5:/> routes ISP_2
Routing table: ISP_2
Flags  Network            Iface          Gateway         Local IP        Metric
------ ------------------ -------------- --------------- --------------- ------
       10.10.20.0/24      G2                                             100
       192.168.10.0/24    Vlan_10                                        100
       0.0.0.0/0          G2             10.10.20.1                      100
Note: Removing the routes from <main> and add them to the <ISP_2> routing table is done by removing the auto added routes option on the G2 and Vlan_10 interfaces then manually add/create them in the secondary routing table. Since we do not want to change the routing table membership on interface G2 and Vlan_10 we must first remove them from <Main> and manually created them in <ISP_2>.

When we are done we will end up with three routes in the <ISP_2> routing table. Two for the external (G2) interface towards the Internet and one for the local / internal network (Vlan_10).

5.3. Creating the needed Policy Based Routing rule.

Now when the route setup is completed we must create a PBR rule that tells the Firewall in what situations this secondary routing table (ISP_2) should be used. Since we want Internet access for our Vlan_10 interface to use ISP2 behind the G2 interface we need to create a PBR rule that looks like this:

005_PBRRule.png
005_PBRRule.png (28.81 KiB) Viewed 266 times

The PBR rule can be read as something like this:

“If traffic is received on the “Vlan_10” interface and is part of the “Vlan10_net” network and is heading towards the G1 interface no matter the destination network and no matter which port or protocol is used, make an override and insert the traffic into the “ISP_2” routing table for further processing.”.

One thing that stands out here in the PBR rule is that the destination interface is G1 and not G2. The reason for this is because the Firewall will perform a route lookup for the destination IP in the <Main> routing table. Therefore outgoing PBR rules destination interface MUST be based on the route lookup for the destination IP in the <Main> routing table.

Since the all-nets route in the <Main> routing table towards the Internet point to G1, the PBR rule’s destination network must also be G1 in order to override it.

Warning: PBR rules are extremely powerful as they will be consulted first before any route lookup, you can override pretty much any route lookup using a PBR rule.

5.4. Modifying the IP policy rules to use the secondary ISP.

The last thing we need to change is the IP policy rules. Since they are currently pointing towards the G1 interface in order to reach the internet we need to change them to point towards the G2 interface instead as that is the primary ISP to be used for users behind the Vlan_10 interface due to the PBR rule “override”.

004.png
004.png (46.1 KiB) Viewed 338 times

5.5. Testing the rules and routes using ping simulations.

Similar do our VR example we first evaluate that Internet access seems to be working by sending a ping to a host on the Internet and simulating that it is generated and received on/from the Vlan_10 interface.

E5:/> ping 8.8.8.8 -srcif=Vlan_10 -srcip=192.168.10.50 -verbose
Rule and routing information for ping:
PBR selected by rule "Vlan10_To_ISP2" - PBR table "ISP_2"
     allowed by rule "Vlan_10_Ping"

Sending 1 4-byte ICMP ping to 8.8.8.8 from 10.10.20.2
 sent via route "0.0.0.0/0 via G2, gw 10.10.20.1" in PBR table "ISP_2"

Ping Results:  Sent: 1, Received:1, Avg RTT: 10.0 ms

The simulation looks good, in the simulation output above we see that the PBR rule triggers and forwards the traffic into the ISP_2 routing table where it performs a new route lookup and finds the destination network (8.8.8.8) on the Internet route on G2.

Next we will try to verify that Vlan_10 is unable to reach the network behind GESW.

E5:/> ping 10.10.10.25 -srcif=Vlan_10 -srcip=192.168.10.50 -verbose
Rule and routing information for ping:
     DROPPED by rule "Default_Access_Rule"

This output looks a bit strange doesn’t it? The reason for this message is basically due to routing. The Firewall does not believe that IP address 192.168.10.50 is routed behind the Vlan_10 interface. The reason why it thinks this is because the interface Vlan_10 is a member of all routing tables. This means that the Firewall will attempt to find 192.168.10.50 in the <Main> routing table and the only matching route for this network in the <Main> routing table is the G1 interface. So the packets will be dropped if we attempt this, and this is exactly what we want it to do in this scenario. We do not want Vlan_10 to be able to reach the GESW network.

To further visualize the routing issue let us do a route lookup in the main routing table to make the Firewall tell us where it believes 192.168.10.50 to be located.

E5:/> route -lookup=192.168.10.50
Looking up 192.168.10.50 in routing table "main":

  Matching route: 0.0.0.0/0
  Routing table : main
  Send via iface: G1
  Gateway       : 192.168.98.1

  Proxy ARP on  :
  Local IP      : (use iface IP in ARP queries)
  Metric        : 100
  Flags         :

As we can see the Firewall believes 192.168.10.50 is routed behind G1 and not Vlan_10 and will drop the packet.

Since we do not have any PBR rules that could possibly override this behavior, the packets will be dropped due to “Default_Access_Rule”. More information about “Default_Access_Rule” can be found in the following FAQ: viewtopic.php?f=18&t=3520

If we try a second ping simulation to try the reverse, to try reach the Vlan_10 network from the GESW interface.

ping 192.168.10.50 -srcip=10.10.10.50 -pbr=ISP_2 -verbose

Sending 1 4-byte ICMP ping to 192.168.10.50 from 10.10.10.50 using PBR table "ISP_2"
 ... using route "192.168.10.0/24 via Vlan_10, no gw" in PBR table "ISP_2"

Ping Results:  Sent: 1, Received:0, Loss: 100%

This output can look a bit confusing as it may actually look like it might be working. The reason for this is due to a limitation in the ping simulation. Since –pbr and –srciface cannot be combined it means that it will not be possible to simulate this fully in another routing table. This means that the packet will be initiate from Core itself and that will always be allowed. Normally this would have been dropped by either “Default_Access_Rule” or the “Default_Drop_Rule” that triggers if no matching IP policy or rule can be found. So this particular test cannot be simulated but rather needs to be tested using real machines.

Based on tests and available data, all requirements has been fulfilled using PBR rules.

Question: Why use VR or PBR if they do the same thing?

Both have their strengths and weaknesses and it may depend on what the administrator actually want to do. But in order to give a VERY brief advantage list:
  • VR is better at segmenting networks without the need for any additional PBR rules or exceptions. Just change the interface membership and you are pretty much done.
  • PBR is better at controlling traffic. With PBR rules you can for instance make a PBR rule trigger only on a specific IP, port or protocol. An example would be if you want to offload DNS and HTTP/HTTPS or maybe VoIP traffic to a secondary ISP, then you need to use PBR because only PBR can redirect traffic based on service/ports.
Question: What about incoming traffic in the PBR scenario?

Incoming traffic can be a bit more difficult as the PBR rule needed for it depends on what the administrator wants to do. But to make a really simple example, we want to allow incoming traffic to the interface IP on G2 (ISP2). It can be for instance that we have a webserver behind Vlan_10 that we want to address translate (SAT) traffic going to G2_ip to a webservers private IP located behind Vlan_10.

First we need to setup an incoming PBR rule that looks like in the below image.

006_PBRRuleIncoming.png
006_PBRRuleIncoming.png (19.75 KiB) Viewed 266 times

Then we need a Policy rule that allows the traffic and forwards it to the webserver behind Vlan_10 as shown in the below image.

007_IncomingPolicy.png
007_IncomingPolicy.png (12.44 KiB) Viewed 266 times

And lastly we test the rule and routes using a ping simulation and it should look something like this:

E5:/> ping 10.10.20.2 -srcif=G2 -srcip=8.8.8.8 -tcp -port=443 -verbose
Rule and routing information for ping:
TCP: 8.8.8.8:14494 -> 10.10.20.2:443 PBR selected by rule "ISP2_To_Vlan10" - PBR table "ISP_2"
     TCP: 8.8.8.8:14494 -> 10.10.20.2:443 allowed by rule "G2_Webserver"

Sending 0-byte TCP ping to 192.168.10.50:443 from 8.8.8.8:14494
 sent via route "192.168.10.0/24 via Vlan_10, no gw" in PBR table "ISP_2"

In order to evaluate if this ping simulation looks good or not (the webserver in this case is not present as we only want to test to verify that the rule and routes seem to be correct) we look at the following output in the ping simulation:

1. The correct PBR rule triggers and inserts it into the ISP_2 routing table.
2. The correct IP policy triggers (G2_Webserver)
3. Since the IP policy “G2_Webserver” is a policy that performs address translation we see that it forwards the packet further to 192.168.10.50 which is then sent out on the Vlan_10 interface, again using the ISP_2 routing table.

Based on the ping simulation output, everything looks to be working as expected and we should be set to use the configuration in a live network.

Post Reply