Posted on by patrickmarcpreuss

Zero Touch Provisioning for HPE / Aruba Network Equipment

Recently, I’ve been tasked with automating the provisioning of certain network equipment. To achieve this, we’ve created the initial configuration using an Ansible playbook in conjunction with a Jinja2 template. This setup allows the DHCP server to instruct the switch to retrieve the configuration file via TFTP, and we’ve also made provisions for firmware updates to be applied as needed.

  1. Workflow
  2. Creating the Config
  3. DHCP Server
    1. HP Aruba Switch Class
    2. tcpdump from a dhcp request
    3. DHCP response
    4. The Class definition
    5. The Final ISC DHCP Class
    6. Force config file download
    7. Some Details from the Journey
      1. DHCP request
      2. DHCP response
      3. Logs
      4. Firewall issues on the Server Side
      5. Not so helpful file not found.
      6. Save the config
      7. Concat with hardware address is not working.
  4. TFTP Service
  5. Resetting the Switch to Factory defaults
  6. Change in DHCP Classe name
  7. Image update
  8. Cisco AP Class

Workflow

Considering the requirement that each site must be provisioned and maintained independently, regardless of connectivity to a data center, and with no internet connection available, we have opted not to depend on a robust backend system such as a CMDB (e.g., ITop or Netbox) initially. Instead, we have chosen to employ a simple Excel sheet for site provisioning.

Creating the Config

Creating the config was kinda straight forward.

DHCP Server


Next, we addressed the need for a DHCP server. To accomplish this, we implemented a Vendor Class based on the specifications provided by HPE/Aruba.

HP Aruba Switch Class

option space HP_Aruba_Switch;

option HP_Aruba_Switch.config-file-name code 144 = text;
option HP_Aruba_Switch.image-file-name code 145 = text;
option HP_Aruba_Switch.CentralOnPrem-FQDN code 146 = text;
option HP_Aruba_Switch-encapsulation code 43 = encapsulate HP_Aruba_Switch;

Asign this to the static entry of the Switch for testing.

 host aruba-switch {
    fixed-address <IP_ADDRESS>;
    option HP_Aruba_Switch.config-file-name "0897347cfffe.cfg";
    option tftp-server-name "192.0.2.254";
    hardware ethernet <MAC_ADDRESS>;
}

tcpdump from a dhcp request

sudo tcpdump -i eth0 -n -vvv -s0 -X port 67 or port 68
198.51.100.10.bootps > 192.0.2.254.bootps: [udp sum ok] BOOTP/DHCP, Request from 08:97:34:7c:ff:fe, length 306, hops 1, xid 0x6bd10164, Flags
[none] (0x0000)
          Gateway-IP 198.51.100.10
          Client-Ethernet-Address 08:97:34:7c:ff:fe
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message (53), length 1: Discover
            MSZ (57), length 2: 9156
            Vendor-Class (60), length 29: "HP J9774A 2530-8G-PoEP Switch"
            Parameter-Request (55), length 11:
              Subnet-Mask (1), Default-Gateway (3), Time-Server (4), TTL (23)
              Vendor-Option (43), Domain-Name-Server (6), Domain-Name (15), Unknown (119)
              NTP (42), Time-Zone (2), Vendor-Class (60)
            Agent-Information (82), length 12:
              Unknown SubOption 150, length 4:
                0x0000:  0aff 0400
              Unknown SubOption 152, length 4:
                0x0000:  0aff 040a
            END (255), length 0

DHCP response 

192.0.2.254.bootps > 198.51.100.2.bootps: [bad udp cksum 0x6f6c -> 0x227a!] BOOTP/DHCP, Reply, length 341, hops 1, xid 0x6bd10164, Flags [none] (0x0000)
          Your-IP 198.51.100.49
          Gateway-IP 198.51.100.2
          Client-Ethernet-Address 08:97:34:7c:ff:fe
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message (53), length 1: Offer
            Server-ID (54), length 4: 192.0.2.254
            Lease-Time (51), length 4: 600
            Subnet-Mask (1), length 4: 255.255.255.0
            Default-Gateway (3), length 4: 198.51.100.1
            Vendor-Option (43), length 18: 144.16.48.56.57.55.51.52.55.99.102.50.56.48.46.99.102.103
            Domain-Name-Server (6), length 8: 192.0.2.254,192.0.2.253
            Domain-Name (15), length 7: "pp52.de"
            Vendor-Class (60), length 29: "HP J9774A 2530-8G-PoEP Switch"
            END (255), length 0

The DHCP Server does not send the TFTP Sever address because it was not requested?

The Class definition

So no lets move the file into the Class and build the filename from the mac address later on.

    class "HP J9774A 2530-8G-PoEP Switch" {
         match if option vendor-class-identifier = "HP J9774A 2530-8G-PoEP Switch";
         option vendor-class-identifier "HP J9774A 2530-8G-PoEP Switch";
         option tftp-server-name "192.0.2.1";
         option HP_Aruba_Switch.config-file-name "0897347cfffe.cfg";

    }

Next we did some testing with the harware address, sure this got us some strange filename s. 

class "HP J9774A 2530-8G-PoEP Switch" {
         match if option vendor-class-identifier = "HP J9774A 2530-8G-PoEP Switch";
         option vendor-class-identifier "HP J9774A 2530-8G-PoEP Switch";
         option tftp-server-name "192.0.2.1";
          option HP_Aruba_Switch.config-file-name = concat( hardware, ".cfg");

    }

tcpdump output:
<code>TFTP, length 36, RRQ "^A^HM-^W4|M-rM-^@.cfg" netascii blksize 141</code>

The Final ISC DHCP Class

So we our final class looks like

option space HP_Aruba_Switch;
option HP_Aruba_Switch.config-file-name code 144 = text;
option HP_Aruba_Switch.image-file-name code 145 = text;
option HP_Aruba_Switch.CentralOnPrem-FQDN code 146 = text;
option HP_Aruba_Switch-encapsulation code 43 = encapsulate HP_Aruba_Switch;

class "HP J9774A 2530-8G-PoEP Switch" {
         match if option vendor-class-identifier = "HP J9774A 2530-8G-PoEP Switch";
         option vendor-class-identifier "HP J9774A 2530-8G-PoEP Switch";
         option tftp-server-name "192.0.2.254";
         # result will look like 0897347cfffc.cfg


         option HP_Aruba_Switch.config-file-name = concat (
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,1,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,2,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,3,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,4,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,5,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,6,1))),2),
                ".cfg"
         );
 
    }

Force config file download

With dhcp config-file-update set the Switch will ask for an config file. 

Oct 2 15:17:18 198.51.100.49 00090 dhcp: Trying to download Config File (using TFTP) received in DHCP from 192.0.2.254
Oct 2 15:17:18 198.51.100.49 00131 tftp: Transfer completed
Oct 2 15:17:21 198.51.100.49 04331 mgr: syslog: Information logging started on the SYSLOG server 192.0.2.254 over UDP protocol
Oct  2 15:17:21 198.51.100.49 00156 update:  Finished, identical file, no update performed.

So the HP switch is now loading the configuration from the TFTP Server during reboots.

Some Details from the Journey

Debugging DHCP with dhcpdump
dhcpdump -i eno1 -h 08:97:34:7c:f2:80
---------------------------------------------------------------------------

  TIME: 2023-10-03 07:49:50.325
    IP: 192.0.2.254 (0:ff:ff:22:53:51) > 198.51.100.3 (0:0:ff:0:1:a)
    OP: 2 (BOOTPREPLY)
 HTYPE: 1 (Ethernet)
  HLEN: 6
  HOPS: 1
   XID: 6bd10e38
  SECS: 0
 FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 198.51.100.49
SIADDR: 0.0.0.0
GIADDR: 198.51.100.3
CHADDR: 08:97:34:7c:ff:fc:00:00:00:00:00:00:00:00:00:00
 SNAME: .
 FNAME: .
OPTION:  53 (  1) DHCP message type         5 (DHCPACK)
OPTION:  54 (  4) Server identifier         192.0.2.254
OPTION:  51 (  4) IP address leasetime      86400 (24h)
OPTION:   1 (  4) Subnet mask               255.255.255.0
OPTION:   3 (  4) Routers                   198.51.100.1
OPTION:  66 ( 13) TFTP server name          192.0.2.254
OPTION:  43 ( 18) Vendor specific info      9010303839373334 ..089734
                                            3763663238302e63 7cfffc.c
                                            6667             fg
OPTION:   6 (  8) DNS server                192.0.2.254,192.0.2.253
OPTION:  15 ( 10) Domainname                pp52.de
OPTION:  60 ( 29) Vendor class identifier   HP J9774A 2530-8G-PoEP Switch
More details

DHCP request

 Parameter-Request (55), length 13:
 Subnet-Mask (1), Default-Gateway (3), Time-Server (4), TTL (23)
              BF (67), TFTP (66), Vendor-Option (43), Domain-Name-Server (6)
              Domain-Name (15), Unknown (119), NTP (42), Time-Zone (2)
              Vendor-Class (60)

DHCP response

TFTP (66), length 13: "192.0.2.254"

Logs

Oct 2 13:59:26 198.51.100.49 00090 dhcp: Trying to download Config File (using TFTP) received in DHCP from <span style="background-color: initial;font-size: inherit">192.0.2.254</span>
Oct 2 13:59:38 198.51.100.49 00136 tftp: Connection to <span style="background-color: initial;font-size: inherit">192.0.2.254</span> failed

Oct 2 14:09:02 198.51.100.49 05101 amp-server: AMP server configuration is disabled due to first configuration.

Firewall issues on the Server Side

Oct 2 14:29:29 server in.tftpd[81299]: nak: Operation not permitted

Not so helpful file not found.

Oct 2 14:42:23 10.255.4.49 00132 tftp: RCVD error:1, msg: File not found.

Save the config

copy startup-config tftp 198.51.100.49 HP-2530-8G-PoEP.
cfg
TFTP download in progress.

Concat with hardware address is not working.

tcpdump revieals we are getting garbage,

TFTP, length 36, RRQ "^A^HM-^W4|M-rM-^@.cfg" netascii blksize 141

TFTP Service

Place the files in /var/lib/tftpboot.
access needs at least “o+r” others read

Details 

vi /usr/lib/systemd/system/tftp.service
[Service]
ExecStart=/usr/sbin/in.tftpd -s /var/lib/tftpboot -c
StandardInput=socket
# systemctl daemon-reload
# systemctl restart tftp
# grep tftp /var/log/messages
Oct 2 15:18:19 server in.tftpd[81949]: Client ::ffff:198.51.100.49 finished 0897347cf280.cfg

Resetting the Switch to Factory defaults

HP-2530-8G-PoEP# erase all zeroize
erase all zeroize 

HP-2530-8G-PoEP# erase all zeroize
The system will be rebooted and all management module files except
software images will be erased and zeroized. This will take up to 60
minutes and the switch will not be usable during that time.

Continue (y/n)? y
BfocR
ROM information:
   Build directory: /ws/swbuildm/rel_lakesrom_qaoff/rom/build/lakesrom(swbuildm_rel_lakesrom_qaoff_rel_lakesrom)
   Build date:      Aug 20 2017
   Build time:      22:15:54
   Build version:   YA.15.20
   Build number:    10016



Booting Primary Software Image...

Decompressing...done.




Initializing...

ATTENTION: Zeroization has started and will take up to 60 minutes.
           Interrupting this process may cause the switch
           to become unusable.

Backing up firmware images and other system files...
Zeroizing the file system... 100%
Verifying cleanness of the file system... 100%
Restoring firmware images and other system files...
Zeroization of the file system completed.
Continue initializing.
Initialization done

Waiting for Speed Sense.  Press <Enter> twice to continue.



Connected at 9600 baud
HP J9774A 2530-8G-PoEP Switch
Software revision YA.16.05.0008

 (C) Copyright 2018 Hewlett Packard Enterprise Development LP

                      RESTRICTED RIGHTS LEGEND
 Confidential computer software. Valid license from Hewlett Packard Enterprise
 Development LP required for possession, use or copying. Consistent with FAR
 12.211 and 12.212, Commercial Computer Software, Computer Software
 Documentation, and Technical Data for Commercial Items are licensed to the
 U.S. Government under vendor's standard commercial license.

We'd like to keep you up to date about:
  * Software feature updates
  * New product announcements
  * Special events
Please register your products now at:  www.hpe.com/networking/register


System went down:  10/03/23 09:08:48
Saved crash information:
Operator cold reboot from CONSOLE session.

Press any key to continue
You have never previously logged in.
erase all 
HP-2530-8G-PoEP# erase all
The system will be rebooted and all management module files except
software images will be erased.

Continue (y/n)? y
BfocR
ROM information:
   Build directory: /ws/swbuildm/rel_lakesrom_qaoff/rom/build/lakesrom(swbuildm_rel_lakesrom_qaoff_rel_lakesrom)
   Build date:      Aug 20 2017
   Build time:      22:15:54
   Build version:   YA.15.20
   Build number:    10016



Booting Primary Software Image...

Decompressing...done.




Initializing...
Initialization done

Waiting for Speed Sense.  Press <Enter> twice to continue.



Connected at 9600 baud
HP J9774A 2530-8G-PoEP Switch
Software revision YA.16.05.0008

 (C) Copyright 2018 Hewlett Packard Enterprise Development LP

                      RESTRICTED RIGHTS LEGEND
 Confidential computer software. Valid license from Hewlett Packard Enterprise
 Development LP required for possession, use or copying. Consistent with FAR
 12.211 and 12.212, Commercial Computer Software, Computer Software
 Documentation, and Technical Data for Commercial Items are licensed to the
 U.S. Government under vendor's standard commercial license.

We'd like to keep you up to date about:
  * Software feature updates
  * New product announcements
  * Special events
Please register your products now at:  www.hpe.com/networking/register


System went down:  10/03/23 09:21:51
Saved crash information:
Operator cold reboot from CONSOLE session.

Press any key to continue
You have never previously logged in.


HP-2530-8G-PoEP# show log
 Keys:   W=Warning   I=Information
         M=Major     D=Debug E=Error
----  Event Log listing: Events Since Boot  ----
I 01/01/90 00:00:25 03802 chassis: System Self test started on  Stand alone CPU
I 01/01/90 00:00:27 03803 chassis: System Self test completed on  Stand alone
            CPU
I 01/01/90 00:00:29 00061 system: -----------------------------------------
I 01/01/90 00:00:29 00063 system: System went down:  10/03/23 09:21:51
M 01/01/90 00:00:29 00064 system: Operator cold reboot from CONSOLE session.
I 01/01/90 00:02:53 00646 ssh: New 2048-bit RSA SSH host key installed.
I 01/01/90 00:02:54 00092 dhcp: Enabling Auto Image Config Download via DHCP and
            turning off auto-tftp if enabled
I 01/01/90 00:02:54 05177 ip: Setting IP address 0.0.0.0 as default gateway.
I 01/01/90 00:02:54 00690 udpf: DHCP relay agent feature enabled
I 01/01/90 00:02:54 02637 srcip: TACACS admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: TACACS oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: RADIUS admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: RADIUS oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: SYSLOG admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: SYSLOG oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: TELNET admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: TELNET oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: TFTP admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: TFTP oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: SNTP admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: SNTP oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: SFLOW admin policy is 'outgoing interface'
I 01/01/90 00:02:54 02638 srcip: SFLOW oper policy is 'outgoing interface'
I 01/01/90 00:02:54 02637 srcip: RADIUS admin policy for IPv6 is 'outgoing
            interface'
I 01/01/90 00:02:54 02638 srcip: RADIUS oper policy for IPv6 is 'outgoing
            interface'
I 01/01/90 00:02:54 00110 telnet: telnetd service enabled
I 01/01/90 00:02:54 00399 stack: Stack Protocol enabled
I 01/01/90 00:02:54 00433 ssh: Ssh server enabled
I 01/01/90 00:02:54 03365 auth: Creating user last login file
I 01/01/90 00:02:54 04695 auth: Command authorization method set to none.
M 01/01/90 00:02:54 00243 snmp: SNMP changed Engine ID to
            0000000b00000897347cf281.  The user table may need to re-entered.
I 01/01/90 00:02:54 00128 tftp: Enable succeeded
I 01/01/90 00:02:54 00417 cdp: CDP enabled
I 01/01/90 00:02:54 00688 lldp: LLDP - enabled
I 01/01/90 00:02:54 02633 SNTP: Client authentication is disabled.
I 01/01/90 00:02:54 00410 SNTP: Client is enabled.
I 01/01/90 00:02:55 00066 system: System Booted
I 01/01/90 00:02:58 00076 ports: port 1 is now on-line
I 01/01/90 00:04:02 00179 mgr: SME CONSOLE Session - MANAGER Mode
I 01/01/90 00:04:07 00083 dhcp: updating IP address and subnet mask
I 01/01/90 00:04:07 05177 ip: Setting IP address 10.255.4.1 as default gateway.
I 01/01/90 00:04:07 00025 ip: DEFAULT_VLAN: ip address 10.255.4.49/24 configured
            on vlan 1
I 01/01/90 00:04:07 03783 dhcp: DHCP server did not offer all the DNS parameters
            on Primary VLAN
I 01/01/90 00:04:07 05224 activate: Zero-touch provisioning enabled; connecting
            to Aruba Activate server to provision system.
I 01/01/90 00:04:07 05225 activate: Loading security certificates and
            synchronizing time.
I 01/01/90 00:04:07 05625 activate: Trying to sync switch time with NTP server.
I 01/01/90 00:04:08 00091 dhcp: Trying to download Image File (using TFTP)
            received in DHCP from 10.255.83.254
----  Bottom of Log : Events Listed = 47  ----
HP-2530-8G-PoEP#
HP-2530-8G-PoEP#
Rebooting switch...

BfocR
ROM information:
   Build directory: /ws/swbuildm/rel_lakesrom_qaoff/rom/build/lakesrom(swbuildm_rel_lakesrom_qaoff_rel_lakesrom)
   Build date:      Aug 20 2017
   Build time:      22:15:54
   Build version:   YA.15.20
   Build number:    10016

Change in DHCP Classe name

OPTION: 60 ( 42) Vendor class identifier HP J9774A 2530-8G-PoEP Switch dslforum.org

    class "HP J9774A 2530-8G-PoEP Switch dslforum.org" {
         match if option vendor-class-identifier = "HP J9774A 2530-8G-PoEP Switch dslforum.org";
         option vendor-class-identifier "HP J9774A 2530-8G-PoEP Switch";
         option tftp-server-name "192.0.2.254";
         option HP_Aruba_Switch.config-file-name = concat (
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,1,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,2,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,3,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,4,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,5,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,6,1))),2),
                ".cfg"
         );
         log (info, (concat("01 Filename: ", cfgfilename)));
    }

The switch will need a few minutes process the changes. This will involve a few reboots.

Image update

At last add the option to do also the Image update.

    class "HP J9774A 2530-8G-PoEP Switch dslforum.org" {
         match if option vendor-class-identifier = "HP J9774A 2530-8G-PoEP Switch dslforum.org";
         option vendor-class-identifier "HP J9774A 2530-8G-PoEP Switch dslforum.org";
         option tftp-server-name "192.0.2.254";
         option HP_Aruba_Switch.config-file-name = concat (
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,1,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,2,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,3,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,4,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,5,1))),2),
                suffix (concat ("0", binary-to-ascii (16, 8, "", substring(hardware,6,1))),2),
                ".cfg"
         );
        option HP_Aruba_Switch.image-file-name = "HP-J9774A-2530-8G-PoEP-Switch-current.img";
         log (info, (concat("01 Filename: ", cfgfilename)));
    }

Does it work in the DSL class ?

dhcpdump output
---------------------------------------------------------------------------

  TIME: 2023-10-03 11:21:07.254
    IP: 192.0.2.254 (0:ff:4c:22:53:fc) > 198.51.100.2 (0:ff:5e:0:1:a)
    OP: 2 (BOOTPREPLY)
 HTYPE: 1 (Ethernet)
  HLEN: 6
  HOPS: 1
   XID: 6bd15d57
  SECS: 28
 FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 198.51.100.49
SIADDR: 0.0.0.0
GIADDR: 198.51.100.2
CHADDR: 08:97:34:7c:ff:fc:00:00:00:00:00:00:00:00:00:00
 SNAME: .
 FNAME: .
OPTION:  53 (  1) DHCP message type         5 (DHCPACK)
OPTION:  54 (  4) Server identifier         192.0.2.254
OPTION:  51 (  4) IP address leasetime      86400 (24h)
OPTION:   1 (  4) Subnet mask               255.255.255.0
OPTION:   3 (  4) Routers                   198.51.100.1
OPTION:  66 ( 13) TFTP server name          192.0.2.254
OPTION:  43 ( 61) Vendor specific info      9010303839373334 ..089734
                                            3763663238302e63 7cfffc.c
                                            6667912948502d4a fg.)HP-J
                                            39373734412d3235 9774A-25
                                            33302d38472d506f 30-8G-Po
                                            45502d5377697463 EP-Switc
                                            682d63757272656e h-curren
                                            742e696d67       t.img
OPTION:   6 (  8) DNS server                192.0.2.254,192.0.2.253
OPTION:  15 ( 10) Domainname                pp52.de
OPTION:  60 ( 42) Vendor class identifier   HP J9774A 2530-8G-PoEP Switch dslforum.org
---------------------------------------------------------------------------

Cisco AP Class

option space Cisco_LWAPP_AP;
option Cisco_LWAPP_AP.server-address code 241 = array of ip-address;

class "Cisco AP c2700" {
         match if option vendor-class-identifier = "Cisco AP c2700";
         option vendor-class-identifier "Cisco AP c2700";
         vendor-option-space Cisco_LWAPP_AP;
         option Cisco_LWAPP_AP.server-address 192.0.2.1;
 }

Networks for Documentation

The blocks
192.0.2.0/24 (TEST-NET-1),
198.51.100.0/24 (TEST-NET-2) and
203.0.113.0/24 (TEST-NET-3)
are provided for use in documentation.

Links

Photo by Taylor Vick on Unsplash

Blog at WordPress.com.