Contents

Introduction

Welcome to the world of CDRouter! And congratulations on purchasing the leading test tool for CPE and access routers!

This guide will get you started with CDRouter and the CDRouter Multiport add-on and will help you through each of the following steps:

  • Identifying hardware and software requirements
  • Installation
  • Setup and test execution using BuddyWeb, the web-based graphical interface
  • Setup and test execution using buddy, the command line interface

If at any time you have additional questions or would like additional assistance, please contact support@qacafe.com.

Hardware and Software Requirements

To get started with CDRouter, you will need a dedicated PC running Linux, the installer, a license file, and a typical SOHO router (referred to as the router under test). Two variations of the basic test setup for CDRouter are shown below. One test setup shows a basic wired LAN connection to the router under test, while the other shows a wireless LAN connection to the router under test.


Wired setup.png Wireless setup.png


We recommend any modern off-the-shelf PC with a Fedora, openSUSE, Debian, or Ubuntu installation. The Linux host PC must also have TCL installed, and a minimum of three installed and addressable network interfaces, two dedicated to CDRouter and one dedicated to your own lab or corporate network connection. For more information on CDrouter’s hardware and software requirements, please see this Knowledge Base article.

The CDRouter Multiport add-on allows multiple wired and/or wireless LAN and multiple wired WAN connections on the router under test to be connected and tested simultaneously. If you are using CDRouter Multiport and wish to test multiple LAN and/or WAN interfaces on the router under test, you will need additional network interfaces installed in the Linux host PC. One dedicated network interface per LAN or WAN that you wish to test is required.

CDRouter and CDRouter Multiport are similar in many respects and will be referenced simply as ‘CDRouter’ throughout this guide. Additional information applicable to CDRouter Multiport will be highlighted as necessary.

Notes

  1. For information on the network interface cards that we recommend, please refer to this Knowledge Base article.
  2. The network interfaces used for CDRouter should be configured with a static IP address of 0.0.0.0, as discussed in this Knowledge Base article.
  3. The CDRouter Demo requires that Tcl be installed on the Linux host PC. Most distributions install Tcl by default, however, some do not. Instructions for installing Tcl can be found in this Knowledge Base article.
  4. If the router under test is a DSL or DOCSIS based device, CDRouter can be still be used, provided a DSLAM or CMTS, respectively, is available. For more information, please refer to these Application Notes:
    1. DSL CPE Testing with CDRouter
    2. DOCSIS Cable Gateway Testing with CDRouter

Physical Connections

Connect the router under test to CDRouter as shown in the applicable test setup above. Be sure that you know which network interface on the Linux host PC is connected to each port on the router under test. For example, network interface eth1 may be connected to one of the LAN ports while eth2 may be connected to the WAN port of the router under test. This information will be required later during the configuration process. If you are using the CDRouter Multiport add-on and plan to test multiple LAN and/or WAN interfaces on the router under test, be sure to note the names of all of the interfaces that will be used (for example, eth1 and eth2 are both connected to LAN interfaces on the router under test, while eth3 and eth4 are both connected to WAN interfaces).

Installation

Once a suitable Linux host PC has been identified, CDRouter or the CDRouter Multiport add-on can be installed in a few simple steps. You may need to log in to the private portion of our customer web site, referred to as the Customer Lounge, to complete installation. If you do not already have a customer login and password, please contact support@qacafe.com.

Click Here to view CDRouter Installation Instructions

Setup and Test Execution Using the Web Interface (BuddyWeb)

CDRouter includes a comprehensive, multi-user, web based interface for management and control of QA Cafe test suites including CDRouter and the CDRouter Multiport add-on, also known as BuddyWeb. BuddyWeb includes all of the functionality found in buddy and simplifies many of the tasks associated with running and managing test runs. BuddyWeb allows you to easily configure, execute, and view the results of test runs right from a web browser. These results can be easily shared, exported, and imported directly from a web browser as well.

BuddyWeb allows you to:

  • Create test “packages” to capture configuration, test selection, and run time options
  • Initiate test runs via the web or command-line
  • View "live" results while tests are running or view archived results
  • Bundle together all log and capture file associated with a single test run
  • View capture files and packet decodes within the browser without having to use Wireshark
  • Share test results with engineers and managers
  • Import and export configuration files, packages, and test results across CDRouter systems

BuddyWeb is included as part of the installation process of CDRouter and the CDRouter Multiport add-on. When BuddyWeb is running, a web server is created on the host system allowing connections from any web browser running locally or on a remote networked system.

Note that BuddyWeb only supports the Firefox, Safari, and Chrome web browsers. Other web browsers may not work correctly with BuddyWeb and are not supported. A minimum screen resolution of 1280x1024 is recommended for all BuddyWeb users.

Connect to BuddyWeb

You can connect to BuddyWeb and begin running tests with CDRouter or the CDRouter Multiport add-on using the default BuddyWeb port of 8015. For example, if your Linux host has an IP address of 10.0.0.5, point your Firefox browser to URL http://10.0.0.5:8015. If you are running locally, you can also use URL http://127.0.0.1:8015, or http://localhost:8015.

Once connected you should see the following BuddyWeb Home page.


BW home.png


BuddyWeb provides several navigation “tabs” at the top of each page:


BW navbar 62.png


Use these tabs to navigate through the BuddyWeb interface. In three simple steps you will be able to configure and execute a set of tests using BuddyWeb.

Create a configuration file using BuddyWeb

Before a test can be run a configuration file must first be created. As mentioned previously, a configuration file is comprised of parameters called testvars. The testvars are used to describe the configuration of the router under test and also control several aspects of CDRouter.

Configuration files can be created, edited, deleted, or imported using the BuddyWeb Configuration Manager. Using the navigation tabs at the top of the screen, click on the Configurations link to access the BuddyWeb Configuration Manager. By default CDRouter and the CDRouter Multiport add-on are loaded with two configuration files named Example_config_1.conf and Example_config_2.conf as shown in the next screenshot.


BW config.png


To edit an existing configuration file, simply click its row or select it from the list in the Configuration Manager and click the “Edit” button in the toolbar at the top. The selected configuration file(s) will open for editing in a new window. Note that the configuration files created using BuddyWeb are stored in the /usr/buddyweb/configs directory and are identical to those created using buddy.

To create a new configuration file, simply click the “New Config” button to edit our default configuration file template, or launch our Configuration Wizard by clicking the “Wizard” button. The Configuration Wizard is graphical configuration file editor with inline help, as shown in the screenshot below. We recommend starting with the Configuration Wizard, which is an intuitive graphical configuration file editor with inline help, as shown in the screenshot below.

Please refer to the Create a new working directory and configuration file section for more information on manually editing configuration files.


BW config2.png


When building an initial configuration file using the Configuration Wizard, it is essential that the Basic LAN and Basic WAN Configuration sections be completely filled out. The fields in these two sections specify the settings for the basic LAN-side and WAN-side connections between CDRouter and the router under test. Clicking on a field name will display help for that particular field in the lower right hand corner of the Configuration Wizard window.

Ultimately the Basic LAN and Basic WAN Configuration sections in the Configuration Wizard must match the configuration of the router under test. It is typically easiest, at least initially, to modify the settings on the router under test to match the default configuration settings in the Configuration Wizard. In cases where this is not possible, be sure to modify any fields appropriately to match the configuration on the router under test. Once you’ve built an initial configuration file, click the “Generate Config” button in the Configuration Wizard toolbar to save it.

Note that clicking the “Generate Config” button will generate the raw configuration file used by CDRouter. You can later edit this file directly or build a new configuration using the Wizard.

Create and execute a test package using BuddyWeb

You are now ready to create a new buddy test package using the BuddyWeb Package Manager. Using the navigation tabs at the top of the screen, click on the Packages link to access the BuddyWeb Package Manager. A package defines all of the components required to execute a group of tests, including the configuration file that should be used, the modules and individual test cases that should be run, and various run-time options (run all tests once, repeat, etc.). Like the Configuration Manager, the Package Manager allows you to create, edit, and delete test packages, making it easy to compile and manage a comprehensive library of different test scenarios.

CDRouter and the CDRouter Multiport add-on are preloaded with the test packages Example package and Example package 2 as shown in the following screenshot.


BW packages.png


To create a new package, simply click the “New Package” button from within the Package Manager. To view and/or edit the contents of an existing package, click anywhere in its row and then click the “Edit” button, as shown in the following screenshot.


BW packages2.png


Clicking the Edit button as shown above will open the selected package in the current window. The package name, description, associated configuration file, tags, options, and test modules can all be modified from the package edit window, as shown in the screenshot below. For more information on tags, which are user-defined labels that can be applied to configuration files, packages, and results, please see this Application Note.

Once a package has been created, it can be executed one of two possible ways:

  • By navigating to the Packages page, selecting one or more test packages from the list and clicking the “Run/Queue” button. All selected packages will be added to the job queue and executed sequentially.
  • By navigating to the Packages page, opening a single package and clicking the “Launch” button. Note that this will automatically redirect you to the Live page.


BW packages3.png

Analyze test results using BuddyWeb

Whenever a test package is executed, all of the individual test cases included in that package are added to the test queue. BuddyWeb’s Live page displays information in real-time about the package and tests currently in the queue, along with links to detailed log files and packet level capture files on both the WAN and LAN interfaces for all completed tests.

A summary of all completed test runs are shown in BuddyWeb’s Results page. From the Results page you can view the detailed results, open the results folder, print a report, or export the results directory for a particular test package. The Results page also displays the number of failures, the number of passes, the test duration, and the size of the results directory, as shown in the screenshot below:


BW results.png


Clicking on any test run will display the results for all individual test cases that were executed. The following screenshot displays the results of pre-loaded “Example package”. As you can see, 21 tests were selected and 10 passed, zero failed, and eleven were skipped. Note that a green checkmark represents a PASS, whereas a red ‘X’ represents a FAIL and a yellow yield represents a SKIPPED test. Tests may be skipped if the requested functionality or specific options are not enabled in the associated configuration file.


BW results2.png


The automatically generated log files provide detailed information that can be used to debug failures. BuddyWeb also automatically captures all packets on the LAN and WAN interfaces and makes these available as capture files. These capture files, and individual packets within a test log, can be viewed directly with BuddyWeb's CloudShark based integrated capture file viewer or alternatively with Wireshark. If Wireshark is used, you can associate your browser with the capture files by linking files of these types to the location of your wireshark executable on disk. This is typically ‘/usr/bin/wireshark’. Also note that log and capture files can be viewed and automatically refreshed while a test case is running.

The router under test, if configured properly, should produce results similar to those shown above. You can examine the results in detail by reviewing the log and capture files.

Setup and Test Execution Using the Command Line Interface (buddy)

CDRouter and the CDRouter Multiport add-on ship with a comprehensive and powerful command line interface known as ‘buddy’. Buddy allows you to fully configure and run CDRouter in a few simple steps. Buddy also includes a number of advanced test execution options, and comprehensive logging, capture, and decode capabilities at both the protocol and packet levels.

This section will provide you with an overview of buddy’s features and capabilities, and will walk you through the process of configuring and executing a basic set of tests using buddy. If you have questions about any of buddy’s advanced configuration or test execution options, issue the buddy -help command and refer to the CDRouter User Guide.

If you plan to use CDRouter’s web-based graphical user interface exclusively, please skip this section.

Create a new working directory and configuration file

To get started with buddy, create a new working directory and copy the example CDRouter configuration file. CDRouter and the CDRouter Multiport add-on are located in the /usr/share/doc/cdrouter directory on your Linux host. The default configuration file for CDRouter and the CDRouter Multiport add-on is also located in this directory and is named ‘local.conf’.

Example:

# mkdir firstTest
# cd firstTest
# cp /usr/share/doc/cdrouter/local.conf .
# cp local.conf firstTest.conf

Before a test can be executed a configuration file must first be created. A configuration file is a list of parameters called testvars. Testvars are used to describe the configuration of the router under test and also control several aspects of CDRouter. Configuration files are text files and can be edited with any text editor. To get started, open the newly created configuration file with your favorite text editor.

# vi firstTest.conf

As you will see, a buddy configuration file consists of many different testvars. There are certain testvars however that are essential to the fundamental test setup and that must be configured before any tests can be performed. A list of the essential testvars required to setup the fundamental LAN- side and WAN-side connections between CDRouter and the router under test are provided in the tables below.


LAN-side Testvar Description Default
lanInterface The name of the physical interface on the Linux host connected to the LAN side of the router under test. eth1
lanIp The IP address of the router under test's LAN interface. 192.168.1.1
lanMask The network mask of the router under test's LAN interface. 255.255.255.0


WAN-side Testvar Description Default
wanInterface The name of the physical interface on the Linux host connected to the WAN side of the router under test. If you are using an ATM card for an xDSL setup, ignore this entry and use the wanAtmInterface testvar instead. See the CDRouter User's Guide for more information. eth2
wanIspIp The IP address CDRouter will use on the WAN side of the router under test. 192.168.200.1
wanIspAssignIp The IP address CDRouter will assign to the router using PPPoE, PPPoA, DHCP, PPTP, etc. 192.168.200.2
wanNatIp This is the address the router will use for NAPT connections. Normally, this is the same as the wanIspAssignIp value. 192.168.200.2
wanIspNextIp This is an unused IP address on the same subnet as the wanIspIp. CDRouter will sometimes switch its IP address to this value. It may also assign this value to the router under test during testing. 192.168.200.254
wanIspMask The network mask for the WAN interface. 255.255.255.0
wanMode The mode of operation on the WAN interface of the router under test. PPPoE
pppoeConnectOnDemand The PPPoE mode of the router under test. If the router is configured for connect on demand mode (versus always-on mode), this testvar should be set to "yes". no
pppoeUser The PPPoE user name configured on the router under test. qacafe
pppoePassword The PPPoE password configured on the router under test. qacafe123


Ultimately the testvars listed above must match the configuration of the router under test. It is typically easiest, at least initially, to modify the settings on the router under test to match the default configuration file. In cases where this is not possible, be sure to modify any testvar values to match the configuration on the router under test. We recommend using the search function in your text editor to quickly find and modify the above testvars.

By default, CDRouter and the CDRouter Multiport add-on are configured for a single Ethernet LAN connection to the router under test. If you plan to utilize a wireless LAN connection rather than, or in addition to, an Ethernet connection, there are number of additional testvars that must be configured, as shown below. Note that wireless security is disabled by default.


Wireless LAN Testvars Description Default
lanType The physical interface type used by CDRouter to connect to the LAN side of the router under test. Can be "ethernet", "802.11a", "802.11b", "802.11g", or "802.11n". ethernet
lanSSID The SSID of the router under test's wireless LAN interface. qa-net
wpaMode The version of the WPA supported by the router under test. Can be either "WPA" or "WPA2". disabled*
lanWEPKey WEP key for the router under test's wireless LAN interface. Can be either a 40 or 128 bit key. disabled**

*If WPA is enabled, a number of additional testvars must also be configured, including: wirelessAuthType, wpaCipher, wpaGroupCipher, wpaKey, wpaIncludeRSNCaps, and wpa802dot11draft.
**If WEP is enabled, an additional testvar, lanWEPKeyIndex may also be configured.


If you are using the CDRouter Multiport add-on, additional LAN and WAN interfaces may be configured and tested simultaneously. Each additional interface must be fully configured using all of the same testvars outlined above. The configuration testvars for any additional interfaces are grouped together under the keyword testvar_group.

Example configurations for additional LAN interfaces are provided at the end of the default configuration file. To enable one of these additional LAN interfaces, simply remove the keyword IGNORE from the beginning of the testvar_group line. Note that an additional testvar called useSameLanInterface is also included. If set to ‘yes’ this testvar will cause the CDRouter-Multiport add-on to use only a single LAN interface for all tests (the interface configured as lan1 will be used for all tests). This testvar is disabled by default, causing the CDRouter-Multiport add-on to switch LAN interfaces between tests.

A full description of all testvars can be found in the CDRouter User Guide.

Select the modules and tests to be performed

CDRouter and the CDRouter Multiport add-on contain more than 1400 individual tests (with all add-ons) that are grouped into what are known as test modules. A test module is a group of tests targeting a specific protocol or functionality. To see which modules are installed on your system, issue the buddy -list command. The output of the buddy -list command displays the module name, the total number of tests contained within the module, and a brief description of the module. The buddy -summary command displays test summaries for all tests currently installed. Test summaries for a specific module can be displayed using the buddy -summary option in conjunction with the buddy -module option. All of the CDRouter or CDRouter Multiport add-on modules are located in the /usr/share/doc/cdrouter/ directory.

Example:

# buddy –module nat.tcl –summary

A link to all test case summaries is also included in the Additional Online Resources section of this guide.

Once you’ve identified which modules and test cases you want to perform, the next step is to execute them using buddy’s advanced test execution controls.

Execute the selected tests

To execute a test using buddy, you must specify the test configuration file (using the buddy -config option) and the location of the test modules to be used (using the buddy -testpath option). You may also specify any of buddy’s additional advanced options. By default, buddy will run all modules and tests unless otherwise specified. The buddy -module option can be used to specify certain modules for execution.

When running a test for the first time using buddy, we recommend using the following options:

These options will turn on additional logging information to help debug any setup issues and/or test failures. The buddy -capture option will produce WAN and LAN capture files in the working directory that may be opened using Wireshark.

You must have root-level access to run tests using buddy. To run all tests within a single module use the buddy -module option with the name of the module. The nat.tcl module is a good initial set of tests to verify that the configuration and test setup are working properly. The buddy command to run all of the tests in the nat.tcl module using our ‘frstTest.conf’ configuration file is:

# buddy –config firstTest.conf –testpath /usr/share/doc/cdrouter –module
nat.tcl –trace –pt –capture –logfile myrun.txt

Note that you will be prompted to restart the router under test before proceeding.

Additional Examples:

To run a specific test (use the buddy -execute option):

# buddy –config firstTest.conf –testpath /usr/share/doc/cdrouter –execute cdrouter_nat_1 

TIP: Instead of using the buddy -testpath option, you may set the environment variable BUDDY_TESTPATH:

# export BUDDY_TESTPATH=/usr/share/doc/cdrouter

Analyze the results

Following completion of a module or test, a summary will be displayed showing the total number of tests that were performed, the number of test failures, and the overall test time. If you selected the buddy -logfile option, buddy will also create a detailed test log in the current working directory. Likewise, the buddy -capture option will result in full capture files in .cap format for both the LAN and WAN interfaces. The capture files will also be stored in the current working directory and may be viewed with Wireshark. If you are using the CDRouter Multiport add-on with multiple configured interfaces, separate capture files will be created for each interface.

CDRouter and CDRouter Multiport add-on log files are separated by test, and contain general status information as well as low level protocol information in various degrees of verbosity based on the buddy options selected when the tests were performed. A small portion of a typical log file is shown below. This snippet shows the information that is logged for a single test case (in this case test cdrouter_nat_2, included in the nat.tcl module) when the buddy -pt and buddy -trace options are enabled:

------------------------------------------------------------------------------
Test cdr-mp-56: Outbound UDP connections use NAPT
Module: nat.tcl
Name: cdrouter_nat_2
Description:

   step 1. Initiate an outbound UDP connection (UDP echo)
   step 2. Verify the udp is received on WAN side
   step 3. Verify the IPv4 src address on the WAN side is the router's address
   step 4. Send a return response back to LAN
   step 5. Verify return packet is received on the LAN

   Reference:
   RFC 3022 Traditional IP Network Address Translator (Traditional NAT)

------------------------------------------------------------------------------
INFO(cdr-mp-56): 13:12:10| Starting test cdrouter_nat_2 (56)
INFO(remoteHost): 13:12:10| Starting up UDP echo server on port 2048
INFO(remoteHost): 13:12:10| Accepting UDP connections on port 2048
INFO(cdr-mp-56): 13:12:10| Initiating outbound UDP connection to '3.3.3.3' from LAN side
INFO(lan): 13:12:10| Initiate UDP echo to 3.3.3.3:2048 using src port 14750
O>>>(lan): 13:12:10|     192.168.0.176     3.3.3.3     UDP     14750 > 2048 len 38
INFO(lan): 13:12:10| Waiting up to 10 seconds for UDP packet at port 14750
I<<<(wan): 13:12:10|     192.168.200.2     3.3.3.3     UDP     14750 > 2048 len 38
INFO(remoteHost): 13:12:10| 3.3.3.3:2048 received UDP from 192.168.200.2:14750
INFO(remoteHost): 13:12:10| Responding to UDP echo request from 192.168.200.2:14750, UDP length 38
O>>>(wan): 13:12:10|     3.3.3.3     192.168.200.2     UDP     2048 > 14750 len 38
I<<<(lan): 13:12:10|     3.3.3.3     192.168.0.176     UDP     2048 > 14750 len 38
INFO(lan): 13:12:10| Received UDP response from 3.3.3.3
PASS(cdr-mp-56): 13:12:10| Received UDP response from 3.3.3.3
PASS(cdr-mp-56): 13:12:10| WAN side IP address is NAT address 192.168.200.2
INFO(remoteHost): 13:12:10| Shuting down UDP echo server on port 2048
INFO(remoteHost): 13:12:10| UDP server on port 2048 has shutdown
PASS(cdr-mp-56): 13:12:10| Test cdrouter_nat_2 (56) passed

As you can see from the above log file snippet, each test that is included in the log file is delineated by its test description. Following the test description is a time stamped line-by-line description of the test. Each line begins with a certain specific keyword that denotes that type of information being logged. In this particular case, the ‘INFO’ status descriptor denotes general status or test execution information, whereas the ‘O>>> (lan)’ and ‘I<<< (wan)’ descriptors indicate packet and protocol information. Specifically, packets that are transmitted (or sent out) on a particular interface (on the lan interface for example) are preceded with ‘O>>>(lan)’ in the log file whereas packets that are received into an interface are preceded with ‘I<<< (lan)’. Additionally, ‘PASS’ and ‘FAIL’ descriptors are used to denote the test status in relation to the specific pass/fail criteria that are defined for each test case. Pass/fail criteria are typically listed in the test description for each test case within the log file.

The log file is designed to provide a detailed step-by-step test execution overview, including a description of all protocols and addresses in use, all transmitted packets, all received packets, and the results. You should be able to read the log file and understand exactly what the purpose of a test is, how it was implemented, and what the expected versus actual results are.

In the above example, the purpose of the test is to verify that the router under test supports outbound UDP connections using NAPT. From the log file you can see that a server setup to receive UDP connections on port 2048 is established at address 3.3.3.3 (this address is controlled by the testvar remoteHostIp. A UDP echo to 3.3.3.3:2048 is initiated from the LAN side on port 14750 (from address 192.168.0.176). This packet is received on the WAN interface of CDRouter and contains the correct address as assigned by CDRouter to the WAN interface of the router under test, 192.168.200.2 (this address is controlled by the testvar wanIspAssignIp). At this point, the first of two success metrics for this test has been met. The WAN interface of CDRouter next responds to the echo request; the response is received on the LAN interface of CDRouter with the correct address (3.3.3.3), therefore the second success metric is also met. Both success metrics have been met, resulting in a pass for this test case.

If additional information for a particular test case is required, the individual capture files may also be analyzed. The capture files include protocol and bit-level decodes for all packets transmitted and received on each interface. Buddy also includes additional logging options that can be used to provide additional information directly within the log file.

Additional Resources

The following resources are available online:

Test case summaries:

User guides:

Customer Lounge - software downloads and updates (login required):

Contact information: