Contents

Configuring URL filtering· 1

About URL filtering· 1

URL· 1

URL filtering rule· 1

URL category· 2

URL filtering policy· 2

URL filtering whitelist/blacklist rule· 2

URL filtering mechanism·· 3

URL filtering signature library management 4

Restrictions: Hardware compatibility with URL filtering· 5

Licensing requirements· 6

URL filtering tasks at a glance· 6

Configuring a URL category· 6

Configuring a URL filtering policy· 7

Copying a URL filtering policy or category· 8

Copying a URL filtering policy· 8

Copying a URL filtering category· 8

Applying a URL filtering policy to a DPI application profile· 8

Activating URL filtering policy and rule settings· 9

Managing the URL filtering signature library· 9

Restrictions and guidelines· 9

Scheduling automatic URL filtering signature library update· 9

Triggering an immediate URL filtering signature update· 10

Performing a URL filtering signature manual update· 10

Rolling back the URL filtering signature library· 10

Enabling DPI engine logging· 11

Configuring URL filtering logging for resource access· 11

About URL filtering logging for resource access· 11

Logging access to only resources in the root directories of websites· 11

Disabling logging for access to resources of specific types· 12

Display and maintenance commands for URL filtering· 12

 

Configuring URL filtering

About URL filtering

URL filtering controls access to the Web resources by filtering the URLs that the users visit.

The software supports only the HTTP URL filtering.

URL

A URL is a reference to a resource that specifies the location of the resource on a network and a mechanism for retrieving it. The syntax of a URL is protocol://host [:port]/path/[;parameters][?query]#fragment. Figure 1 shows an example URL.

Figure 1 URL syntax

 

Table 1 describes the fields in a URL.

Table 1 URL field descriptions

Field	Description
protocol	Transmission protocol, such as HTTP.
host	Domain name or IP address of the server where the indicated resource is located.
[:port]	Optional field that identifies the port number of the transmission protocol. If this field is omitted, the default port number of the protocol is used.
/path/	String that identifies the directory or file where the indicated resource is stored. The path is a sequence of segments separated by zero or multiple forward slashes.
[parameters]	Optional field that contains special parameters.
[?query]	Optional field that contains parameters to be passed to the software for querying dynamic webpages. Each parameter is a <key>=<value> pair. Different parameters are separated by an ampersand (&).
URI	Uniform resource identifier that identifies a resource on a network.

 

URL filtering rule

A URL filtering rule matches URLs based on the content in the URI or hostname field.

URL filtering rule type

URL filtering provides the following types of URL filtering rules:

·     Predefined URL filtering rules—Signature-based URL filtering rules. The device automatically generates them based on the local URL filtering signatures. In most cases, the predefined rules are sufficient for URL filtering.

·     User-defined URL filtering rules—Regular expression- or text-based URL filtering rules that are manfully configured.

URL filtering rule matching method

A URL filtering rule supports the following URL matching methods:

·     Exact match by text—Performs an exact text string match on the hostname or URI field of the URL.

¡     If a rule is configured with the host keyword, a URL matches the rule only if it contains a host name exactly the same as the specified text sting. For example, the rule 1 host text abc.com.cn command matches URLs that carry the abc.com.cn hostname, but it does not match URLs carrying the dfabc.com.cn hostname.

¡     If a rule is configured with the uri keyword, a URL matches the rule if it contains a URI that begins with the complete text string in the rule. For example, the rule 2 uri text /sina/news command matches URLs that contain URIs /sina/news, /sina/news/sports, and /sina/news_sports. However, the command does not match URLs that contain URI /sina.

·     Fuzzy match by regular expression—Performs a fuzzy regular expression match on the hostname or URI field of the URL. For example, the rule 3 host regex sina.*cn command matches URLs that carry the news.sina.com.cn hostname.

URL category

URL filtering provides the URL categorization feature to facilitate filtering rule management.

You can classify multiple URL filtering rules to a URL category and specify an action for the category. If a matching rule is in multiple URL categories, the system takes the action for the category with the highest severity level.

URL filtering supports the following types of URL categories:

·     Predefined URL categories.

The predefined URL categories contain the predefined URL filtering rules. Each predefined URL category has a unique severity level in the range of 1 to 999, and a category name that begins Pre-. Predefined URL categories cannot be modified.

The device supports two levels of URL categories: child URL category and parent URL category.

The parent URL category are predefined and contains only predefined child URL categories.

·     User-defined URL categories.

You can manually create URL categories and configure filtering rules for them. The severity level of a user-defined URL category is in the range of 1000 to 65535. You can edit the filtering rules and change the severity level for a user-defined URL category.

URL filtering policy

A URL filtering policy can contain multiple URL categories, and each category has an action defined for packets that match a filtering rule in the category. You can also specify the default action for packets that do not match any filtering rules in the policy. URL filtering actions include drop, permit, block source, reset, redirect, and logging.

URL filtering whitelist/blacklist rule

The device supports using URL-based whitelist and blacklist rules to filter HTTP packets. If the URL in an HTTP packet matches a blacklist rule, the packet is dropped. If the URL matches a whitelist rule, the packet is permitted to pass through.

URL filtering mechanism

URL filtering takes effect after you apply a URL filtering policy to a DPI application profile.

As shown in Figure 2, upon receiving a packet, the device performs the following operations:

1.     The device compares the packet with the object policy rules or security policy rules.

If the packet matches a rule that is associated with a URL filtering policy (through a DPI application profile), the device extracts the URL from the packet.

For more information about security policies and object policies, see Security Configuration Guide

2.     The device compares the extracted URL with the rules in the URL filtering policy.

¡     If the URL matches a rule, the device determines the actions for the packet as follows:

-     If the matching rule is a whitelist rule, the packet is permitted to pass through.

-     If the matching rule is a blacklist rule, the packet is dropped.

-     If the matching rule is a URL filtering rule, the devices takes the actions specified for the URL category to which the rule belongs.

If the URL filtering rule belongs to multiple URL categories, the actions specified for the URL category with the highest severity level apply.

¡     If the URL does not match any rule in the policy, the default action specified for the policy applies. If the default action is not configured, the device permits the packet to pass through.

 

Figure 2 URL filtering mechanism

‌

URL filtering signature library management

The device uses the local URL filtering signature library to identify URLs in the HTTP packets.

You can update the device URL filtering signature library to the most up-to-date version or roll back the library to a version.

Updating the URL filtering signature library

The following methods are available for updating the URL filtering signature library on the device:

·     Automatic update.

The device periodically accesses the company's website and automatically downloads the most up-to-date URL filtering signature file to update its local signature library.

·     Triggered update.

The device downloads the most up-to-date URL filtering signature file from the company's website to update its local signature library immediately you trigger the operation.

·     Manual update.

Use this method when the device cannot connect to the company's website.

You must manually download the most up-to-date URL filtering signature file from the company's website, and then use the file to update the signature library on the device.

Rolling back the URL filtering signature library

If filtering false alarms or filtering exceptions occur frequently, you can roll back the URL filtering signature library to the previous version or to the factory default version.

Restrictions: Hardware compatibility with URL filtering

Hardware series	Model	Product code	URL filtering compatibility
WX1800H series	WX1804H	EWP-WX1804H-PWR-CN	Yes
WX2500H series	WX2508H-PWR-LTE WX2510H WX2510H-F WX2540H WX2540H-F WX2560H	EWP-WX2508H-PWR-LTE EWP-WX2510H-PWR EWP-WX2510H-F-PWR EWP-WX2540H EWP-WX2540H-F EWP-WX2560H	Yes
WX3000H series	WX3010H WX3010H-X WX3010H-L WX3024H WX3024H-L WX3024H-F	EWP-WX3010H EWP-WX3010H-X-PWR EWP-WX3010H-L-PWR EWP-WX3024H EWP-WX3024H-L-PWR EWP-WX3024H-F	Yes: ·     WX3010H ·     WX3010H-X ·     WX3024H ·     WX3024H-F No: ·     WX3010H-L ·     WX3024H-L
WX3500H series	WX3508H WX3510H WX3520H WX3520H-F WX3540H	EWP-WX3508H EWP-WX3510H EWP-WX3520H EWP-WX3520H-F EWP-WX3540H	Yes
WX5500E series	WX5510E WX5540E	EWP-WX5510E EWP-WX5540E	Yes
WX5500H series	WX5540H WX5560H WX5580H	EWP-WX5540H EWP-WX5560H EWP-WX5580H	Yes
Access controller modules	LSUM1WCME0 EWPXM1WCME0 LSQM1WCMX20 LSUM1WCMX20RT LSQM1WCMX40 LSUM1WCMX40RT EWPXM2WCMD0F EWPXM1MAC0F	LSUM1WCME0 EWPXM1WCME0 LSQM1WCMX20 LSUM1WCMX20RT LSQM1WCMX40 LSUM1WCMX40RT EWPXM2WCMD0F EWPXM1MAC0F	Yes

‌

Hardware series	Model	Product code	URL filtering compatibility
WX1800H series	WX1804H WX1810H WX1820H WX1840H	EWP-WX1804H-PWR EWP-WX1810H-PWR EWP-WX1820H EWP-WX1840H-GL	Yes
WX3800H series	WX3820H WX3840H	EWP-WX3820H-GL EWP-WX3840H-GL	No
WX5800H series	WX5860H	EWP-WX5860H-GL	No

‌

Licensing requirements

The URL filtering module requires a license to run on the device. If the license expires, the existing URL filtering signature library is still available but you cannot upgrade the signature library on the device. For more information about licenses, see license management in License Management Configuration Guide.

URL filtering tasks at a glance

To configure URL filtering:

1.     Configuring a URL category

2.     Configuring a URL filtering policy

3.     (Optional.) Copying a URL filtering policy or category

4.     Applying a URL filtering policy to a DPI application profile

5.     Activating URL filtering policy and rule settings

6.     Managing the URL filtering signature library

7.     (Optional.) Enabling DPI engine logging

8.     (Optional.) Configuring URL filtering logging for resource access

Configuring a URL category

About this task

Perform this task to create a user-defined URL category and configure filtering rules for it to meet specific URL filtering requirements.

Restrictions and guidelines

When creating a URL category, you must assign a unique severity level in the range of 1000 to 65535 to the URL category. The larger the value, the higher the severity level.

Procedure

1.     Enter system view.

system-view

2.     Create a URL category and enter its view.

url-filter category category-name [ severity severity-level ]

By default, the device provides predefined URL categories with names starting with Pre-.

The name of a user-defined URL category cannot start with Pre-.

3.     (Optional.) Configure a description for the URL category.

description text

4.     Configure URL filtering rules for the URL category. Choose the options to configure as needed:

¡     Configure a URL filtering rule.

rule rule-id host { regex regex | text string } [ uri { regex regex | text string } ]

¡     (Optional.) Add the URL filtering rules of a predefined URL category to the URL category.

include pre-defined category-name

By default, a user-defined URL category does not contain the URL filtering rules of any predefined URL category.

5.     (Optional.) Rename the URL category.

rename new-name

Configuring a URL filtering policy

About this task

A URL filtering policy contains the following settings:

·     URL category-to-action mappings.

·     Default action.

·     Whitelist and blacklist rules.

Restrictions and guidelines

If DRS is enabled, the name of a URL filtering policy cannot be drs to avoid unexpected configuration change after reboot. To enable DRS, use the wlan drs enable command. For more information about DRS, see WLAN DRS in WLAN Configuration Guide.

Procedure

1.     Enter system view.

system-view

2.     Create a URL filtering policy and enter its view.

url-filter policy policy-name

3.     Specify the actions for a URL category.

category category-name action { block-source [ parameter-profile parameter-name ] | drop | permit | redirect parameter-profile parameter-name | reset } [ logging [ parameter-profile parameter-name ] ]

By default, no actions are specified for a URL category.

If a packet matches a rule that is in multiple URL categories, the system uses the actions for the category with the highest severity level.

4.     (Optional.) Specify the default action for packets that do not match any rule in the policy.

default-action { block-source [ parameter-profile parameter-name ] | drop | permit | redirect parameter-profile parameter-name | reset } [ logging [ parameter-profile parameter-name ] ]

5.     (Optional.) Configure a whitelist or blacklist rule in the policy.

add { blacklist | whitelist } [ id ] host { regex host-regex | text host-name } [ uri { regex uri-regex | text uri-name } ]

6.     (Optional.) Rename the URL filtering policy.

rename new-name

Copying a URL filtering policy or category

Copying a URL filtering policy

About this task

You can create a new URL filtering policy by copying an existing one.

Procedure

1.     Enter system view.

system-view

2.     Create a URL filtering policy and enter its view.

url-filter copy policy old-name new-name

Copying a URL filtering category

About this task

You can create a new URL category by copying an existing one.

Restrictions and guidelines

When you copy a URL category, be sure to assign a unique severity level to the new URL category.

Procedure

1.     Enter system view.

system-view

2.     Copy a URL category.

url-filter copy category old-name new-name severity severity-level

Applying a URL filtering policy to a DPI application profile

About this task

A URL filtering policy must be applied to a DPI application profile to take effect.

Restrictions and guidelines

A DPI application profile can use only one URL filtering policy. If you apply different URL filtering policies to the same DPI application profile, only the most recent configuration takes effect.

Procedure

1.     Enter system view.

system-view

2.     Enter DPI application profile view.

app-profile app-profile-name

For more information about this command, see DPI engine commands in Security Command Reference.

3.     Assign a URL filtering policy to the DPI application profile.

url-filter apply policy policy-name

By default, no URL filtering policy is applied to the DPI application profile.

Activating URL filtering policy and rule settings

About this task

After you edit the policy and rule settings in the URL filtering module, perform this task to activate the settings.

Restrictions and guidelines

This task can cause temporary outage for all DPI services. As a best practice, perform the task after all DPI service policy and rule settings are complete.

For more information about activating DPI service module configuration, see "Configuring the DPI engine."

Procedure

1.     Enter system view.

system-view

2.     Activate URL filtering policy and rule settings.

inspect activate

By default, URL filtering policy and rule settings do not take effect.

Managing the URL filtering signature library

You can update or roll back the version of the URL filtering signature library on the device.

Restrictions and guidelines

·     Do not delete the /dpi/ folder in the root directory of the storage medium.

·     Do not perform URL filtering signature update and rollback when the device's free memory is below the normal state threshold. For more information about device memory thresholds, see system management in System Management Configuration Guide.

·     For successful automatic and immediate signature update, make sure the device can resolve the domain name of the company's website into an IP address through DNS. For more information about DNS, see DNS configuration in Network Connectivity Configuration Guide.

Scheduling automatic URL filtering signature library update

About this task

You can schedule automatic URL filtering signature library update if the device can access the signature database services on the company's website. The device periodically obtains the latest signature file from the company's website to update its local signature library according to the update schedule.

Procedure

1.     Enter system view.

system-view

2.     Enable automatic URL filtering signature library update and enter automatic URL filtering signature library update configuration view.

url-filter signature auto-update

By default, automatic URL filtering signature library update is disabled.

3.     Schedule the update time.

update schedule { daily | weekly { fri | mon | sat | sun | thu | tue | wed } } start-time time tingle minutes

By default, the device updates the URL filtering signature at a random time between 01:00:00 and 03:00:00 every day.

Triggering an immediate URL filtering signature update

About this task

Anytime you find a release of new signature version on the company's website, you can trigger the device to immediately update the local signature library.

Procedure

1.     Enter system view.

system-view

2.     Trigger an automatic URL filtering signature library update.

url-filter signature auto-update-now

Performing a URL filtering signature manual update

About this task

If the device cannot access the signature database services on the company's website, use one of the following methods to manually update the URL filtering signature library on the device:

·     Local update—Updates the URL filtering signature library on the device by using the locally stored update URL filtering signature file.

(In IRF mode.) Store the update file on the master device for successful signature library update.

·     FTP/TFTP update—Updates the URL filtering signature library on the device by using the file stored on the FTP or TFTP server.

Procedure

1.     Enter system view.

system-view

2.     Manually update the URL filtering signature library on the device.

url-filter signature update file-path

Rolling back the URL filtering signature library

About this task

If a URL filtering signature library update causes exceptions or a high false alarm rate, you can roll back the URL filtering signature library.

Before rolling back the URL filtering signature library, the device backs up the current signature library as the "previous version." For example, the previous library version is V1 and the current library version is V2. If you perform a rollback to the previous version, library version V1 becomes the current version and library version V2 becomes the previous version. If you perform a rollback to the previous version again, the library rolls back to library version V2.

Procedure

1.     Enter system view.

system-view

2.     Roll back the URL filtering signature library to the previous version or to the factory default version.

url-filter signature rollback { factory | last }

Enabling DPI engine logging

About this task

You can enable DPI engine logging for audit purposes. Log messages generated by DPI engine are output to the device information center. The information center then sends the messages to designated destinations based on log output rules. For more information about the information center, see System Management Configuration Guide.

Procedure

1.     Enter system view.

system-view

2.     Enable DPI engine logging.

url-filter log enable

By default, DPI engine logging is disabled.

Configuring URL filtering logging for resource access

About URL filtering logging for resource access

URL filtering logs user access to resources after you specify the logging action for a URL category or as a default action for a URL filtering policy.

You can use either of the following methods to configure URL filtering to log access to specific types of resources:

·     Configure URL filtering to log access to only resources in the root directories of websites.

·     Enable or disable URL filtering logging for access to resources of specific types.

Logging access to only resources in the root directories of websites

1.     Enter system view.

system-view

2.     Configure URL filtering to log only access to resources in the root directories of websites.

url-filter log directory root

By default, URL filtering logs access to Web resources in all directories.

Disabling logging for access to resources of specific types

1.     Enter system view.

system-view

2.     Disable URL filtering logging for access to resources of a specific resource type.

¡     Disable logging for access to resources of a predefined resource type.

url-filter log except pre-defined { css | gif | ico | jpg | js | png | swf | xml }

¡     Disable logging for access to resources of a user-defined resource type.

url-filter log except user-defined text

By default, URL filtering logs access to all resources except for resources of the predefined resource types (including CSS, GIF, ICO, JPG, JS, PNG, SWF, and XML resources).

Display and maintenance commands for URL filtering

Execute display commands in any view and reset commands in user view.

‌

Task	Command
Display URL category information.	display url-filter category [ verbose ]
Display information about the URL filtering signature library.	display url-filter signature information
Display URL filtering statistics.	display url-filter statistics
Clear URL filtering statistics.	reset url-filter statistics

‌