Injecting network latency for testing

[Handruin]

I'm trying to purposefully inject network latency into a specific network for a test I'm running to see how latency is handled by a piece of software. In the past I've used a tool called WANem to get this done on the cheap. The WANem solution replies on adding routes from each computer involved in the test. If I need to add 50ms of latency from Host A to Host B, I set WANem to add 25ms so that the round trip time is around 50ms.

For this to work, I need to run the route add command (or equivalent) for each host OS. My problem now is that I need to test a device that does not have the ability to add routes. I need to test how a specific piece of hardware (Cisco Blade) handles data transfer during a PXE boot in an automated setup.

Is there a way for me to put a piece of software in line of the network communication that will add latency for every packet of data that transfer through it without having to modify each individual server?

 

[Mercutio]

Why can't you just run an additional VM with another instance of WANem? You're doing all static routes anyway, right?

 

[Handruin]

I don't fully understand what you're proposing. Can you explain a little more on your idea? What would the second WANem do, would it route it's own traffic back through the first WANem?

The routes were being defined manually in each OS route table, but now I'm stuck in a situation where I need to inject bi-directional latency between a systems running Redhat Linux and a physical piece of hardware which will PXE boot and install an OS. Normally I can inject latency with WANem when two systems are running an OS because I can route both hosts network traffic through WANem. The physical piece of hardware I'm not able to define a static route through WANem.

If it helps, this is all on a private network and does not need to route anywhere else.

 

[Handruin]

Attached is an example of how the packets are emulated through WANem (this is right from their documentation).

 

[ddrueding]

Can you put the Blade and the Linux server on two different networks, connected by a third PC that has two network cards, and have this machine run WANem? That way it isn't an obfuscated route, it really is the fastest way across.

 

[Handruin]

I could probably do that, even in a virtual environment, but I don't (yet) know if WANem will run traffic through two NICs in the way you described.

While digging, I did find a possible solution to a similar problem here. I don't fully understand what the change does, but it looks like they are recommending I add NAT to one of the NICs and maybe that will allow WANem to route all the traffic and inject latency?

 

[ddrueding]

In theory, you could also just add the route to the linux box, and double the delay.

 

[Handruin]

I did that as my crude initial test. The problem with that is the initial packet was sent and delayed 50ms and the acknowledge packet was returned with no injected latency making it sub 1ms which makes it an odd configuration.

I found some strange anomalies in the performance of the test results and now I want to confirm them. I think the issue is TFTP doesn't handle latency very well at all (supported by research that I found), but to confirm this, I wanted a more accurate test environment.

 

[Mercutio]

I was thinking that you could just use a second WANem session to represent the additional variable delay for your new device. Two WANems in series, I suppose.