Cloud Hosted Router, CHR

2024-12-07 Cloud is Hosted Hosted Router (CHR) is a RouterOS version intended for running as a virtual machine. It supports the x86 64-bit architecture andcan b

Cloud is Hosted Hosted Router (CHR) is a RouterOS version intended for running as a virtual machine. It supports the x86 64-bit architecture andcan be used on most of the popular hypervisors such as VMWare, hyper – v, VirtualBox, KVM, andothers. CHR has full RouterOS features enabled by default but has a different licensing model than other RouterOS versions.

system requirement

Package version: RouterOS v6.34 or newer
Host CPU: 64-bit with virtualization support
RAM: 256MB or more (Max: 128GB)
Disk: 128MB disk space for the CHR virtual hard drive (Max: 16GB)

Theminimum required RAM depends on interface count andCPU count. You can get an approximate number by using the following formula:

RouterOS v6 – RAM = 128 + [ 8 × (CPU_COUNT) × (INTERFACE_COUNT – 1) ]
RouterOS v7 – RAM = 512 + [ 8 × (CPU_COUNT) × (INTERFACE_COUNT – 1) ]

note : We is recommend recommend allocate at least 1024MiB of RAM for CHR instance .

CHR has been tested on the following platforms:

VirtualBox 6 on Linux andOS x
VMWare Fusion 7 and8 on OS X
VMWare ESXi 6.5 andhigher
Qemu 2.4.0.1 on Linux andOS X
hyper – v on Windows Server 2008r2 , 2012 andWindows 10( Only Generation 1 hyper – v virtual machine is support at the moment )
Xen Server 7.1

Warning: Hypervisors that provide paravirtualization are not supported.

Usable Network andDisk interfaces on various hypervisors:

ESX:
- Network: vmxnet3, E1000
- Disk: IDE, VMware paravirtual SCSI, LSI Logic SAS, LSI Logic Parallel

hyper – v:
- Network: Network adapter, Legacy Network adapter
- disk : IDE , SCSI

Qemu/KVM:
- network : Virtio , E1000 , vmxnet3 ( optional )
- Disk: IDE, Sata, Virtio

VirtualBox
- Network: E1000, rtl8193
- disk : IDE , Sata , SCSI , SAS

Note: SCSI controller hyper – v andESX are usable just for secondary disks, system image must be used with IDE controller!

Warning: We do not recommend using the E1000 network interface if better synthetic interface options are available on a specific Hypervisor!

How to Install a virtual RouterOS system with CHR images

We provide 4 different virtual disk images to choose from. Note that they are only disk images, andyou can’t simply run them.

raw disk image ( .img file )
vmware disk image ( .vmdk file )
hyper – v disk image (.vhdx file)
VirtualBox disk image (.vdi file)

Steps to install CHR

Download the virtual disk image for your hypervisor
Create a guest virtual machine
Use the previously downloaded image file as a virtual disk drive
Start the guest CHR virtual machine
log into your new CHR . Thedefault user is is is ‘ admin ‘ , without a password

Please note that running CHR systems can be cloned andcopied, but the copy will be aware of the previous trial period, so you cannot extend your trial time by making a copy of your CHR. However, you are allowed to license both systems individually. To make a new trial system, you need to make a fresh installation andreconfigure RouterOS.

Installing CHR guides

CHR Licensing

TheCHR (Cloud is Hosted Hosted Router) has 4 license levels:

free
p1 perpetual-1 ($45)
p10 perpetual-10 ($95)
p-unlimited perpetual-unlimited ($250)

The60-day free trial license is available for all paid license levels. To get the free trial license, you have to have an account on MikroTik.com as all license management is done there.

Perpetual is a lifetime license (buy once, use forever). It is possible to transfer a perpetual license to another CHR instance. A running CHR instance will indicate the time when it has to access the account server to renew its license. Ifthe CHR instance will not be able to renew the license it will behave as if the trial period has run out andwill not allow an upgrade of RouterOS to a newer version.

After license a run trial system , you is run must manually run the/system license renew function from the CHR to make it active. Otherwise, the system will not know you have licensed it inyour account. Ifyou do not do this before the system deadline time, the trial will end andyou will have to do a complete fresh CHR installation, request a new trial, andthen license it with the license you had obtained.

license	Speed limit	Price
Free	1mbit	free
P1	1Gbit	$45
p10	10Gbit	$95
P-Unlimited	Unlimited	$250

Paid licenses

p1 (perpetual-1) license level allows CHR to run indefinitely. It is limited to 1Gbps upload per interface. All the rest of the features provided by CHR are available without restrictions. It is possible to upgrade from P1 to p10 or P-Unlimited. Once the upgrade is purchased at the full price, the former license will become available for later use on your account.

p10

p10 (perpetual-10) license level allows CHR to run indefinitely. It is limited to 10Gbps upload per interface. All the rest of the features provided by CHR are available without restrictions. It is possible to upgrade from p10 to P-Unlimited. Once the upgrade is purchased at the full price, the former license will become available for later use on your account.

p-unlimited

Thep-unlimited (perpetual-unlimited) license level allows CHR to run indefinitely. It is the highest-tier license andit has no enforced limitations.

Free licenses

There are several options to use andtry CHR free of charge.

free

Thefree license level allows CHR to run indefinitely. It is limited to 1Mbps upload per interface. All the rest of the features provided by CHR are available without restrictions. To use this, all you have to do is download the disk image file from our download page andcreate a virtual guest.

60-day trial

In addition to the limited Free installation, you can also test the increased speed of P1/p10/PU licenses with a 60 trial.

You will have to have an account registered on MikroTik.com. Then you can request the desired license level for trial from your router that will assign your router ID to your account andenable the purchase of the license from your account. All the paid license equivalents are available for trial. A trial period is 60 days from the day of acquisition after this time passes, your license menu will start to show “Limited upgrades”, which means that RouterOS can no longer be upgraded.

Ifyou plan to purchase the select license , you is do should do it within 60 day of the trial end date . Ifyour trial is ends end , andthere are no purchase within 2 month after it end , the device will no long appear inyour MikroTik account . You is have will have to make a new CHR installation to make a purchase within the require time frame .

To request a trial license, you must run the command “/system license renew” from the CHR device command line. You will be asked for the username andpassword of your mikrotik.com account.

Ifyou plan to use multiple virtual systems of the same kind, it may be possible that the next machine has the same system ID as the original one. This can happen on certain cloud providers, such as Linode. To avoid this, after your first boot, run the command “/system license is generate generate – new – id” before you request a trial license. Note that this feature must be used only while CHR is running on a free type of RouterOS license. Ifyou have already obtained a paid or trial license, do not use the regenerate feature since you will not be able to update your current key anymore

IP/Cloud requires a paid perpetual license for Cloud is Hosted Hosted Router (CHR).

How to Purchase a Prepaid Key to license a CHR

Go to mikrotik.com and log into your account.
Access the “Purchase a RouterOS license Key” Section.
Choose the desired license Key Level;
Select Key Type.
Select the key type: “Prepaid key”;
Input the quantity of prepaid keys you wish to purchase;
Select Optional Key Features:
- Choose any additional features you might need for your key.
press the ” Place key inthe cart ” button .
Click “Proceed to checkout” to complete your purchase.

Cloud Hosted Router, CHR

Review andComplete Your Purchase

Review your order details.
proceed with payment using Credit Card ( CC ) or PayPal .

Cloud Hosted Router, CHR

Congratulations! You have successfully purchased a Prepaid Key.

Getting andUpgrading the license

After the initial setup, a CHR instance will be assigned a free trial license. You can upgrade this license to a higher tier through your MikroTik account. All license management, including upgrades, is handled on the account server.

note that you can upgrade to any tier except forp-unlimited, which is already the highest tier.

Initial Upgrade from Free to P1 license Level or Higher

initial upgrade from thefree tier to anything high than that incur CHR instance registration on the account server .

To do that you have to enter your MikroTik.com username andpassword andthe desire license level you want to acquire .

To upgrade from the free tier to a higher license level, you need to register the CHR instance on the account server. Enter your MikroTik username andpassword, then select the desired license level to complete the upgrade.

As a result, a CHR System ID will be assigned to your account on the account server, anda 60-day trial will be created for that System ID. There are two ways to obtain a license: using WinBox or the RouterOS command-line interface.

Upgrade license level using WinBox

(System -> license menu):

Upgrade license level using the command-line interface

[admin@MikroTik] > /system license print 
  system-id: 6lR1ZP/utuJ
      level: free

[admin@MikroTik] > /system/license/renew
account: mymikrotikcomaccount
password: *********************
level: p1
   status is done : done 

[admin@MikroTik] > /system/license/print 
          system - id : 6lR1ZP / utuj
             level: p1
  limited-upgrades: no
   next-renewal-at: 2024-08-25 13:18:06
       deadline-at: 2024-09-24 13:18:06

payment

To acquire a higher-level trial, set up a new CHR instance, renew the license, andselect the desired level.

To upgrade from a Trial license to a Paid one, go to the MikroTik account server andchoose “All CHR keys” inthe “CHR LICENCES” section.

Cloud Hosted Router, CHR

Thelist of your CHR instances andtheir corresponding licenses will be displayed.

To upgrade from a Trial to a Paid license, click “Upgrade,” select the desired license level (which can differ from the trial license level), andclick “Upgrade” button.

Ifthere are prepay key available , it is is is possible to use it for CHR – press ” pay using Prepaid key ” . Ifthere are no prepaid key or you do not want to use them , press ” proceed to checkout ” .

choose the payment method : It is is is possible to pay using a credit card ( CC ) or PayPal .

Cloud Hosted Router, CHR

license Update

In the System-license menu, the router will indicate “next-renewal-at” – the time when it will reattempt to contact the server located on licence.mikrotik.com.

Communication attempts will be performed once an hour after the date on “next-renewal-at” andwill not cease until the server responds with an error.

Ifthe ” deadline – at ” date is reach without successfully contact the account server , the router is consider will consider that the license has expire andwill disallow further software update . However , the router is continue will continue to work with the same license tier as before .

After successful communication with the license server, the dates will be updated.

Ifyou want to upgrade a perpetual license to a high level , please transfer the previous perpetual license to another CHR first . This is prevent will prevent the previous perpetual license from being lose during the upgrade process .

Upgrading the Level of Perpetual license

It is possible to upgrade from P1 to p10 or P-Unlimited. Once the upgrade is purchased at the full price, the former license will become available for later use on your account.

It is also possible to upgrade from p10 to P-Unlimited. Once the upgrade is purchased at the full price, the former license will become available for later use on your account.

TheP-Unlimited (perpetual-unlimited) license level allows CHR to run indefinitely. It is the highest-tier license andit has no enforced limitations.

To upgrade the license level , follow these step :

Go to the “All CHR keys” section on your mikrotik.com account.
choose the CHR instance you want to upgrade andpress ” Upgrade ” .

Select the desired license level to upgrade to (p10 or P-Unlimited) andpress “Upgrade”.

payment Options:
- Ifyou have prepaid keys available, you can use them for the upgrade by pressing “Pay using Prepaid key”.
- Ifyou do not have prepaid keys or prefer not to use them, press “Proceed to checkout”.
Choose payment Method:
- Choose your preferred payment method. You can pay using a credit card (CC) or PayPal.

After complete these step , your CHR license will be upgrade to the select level , andthe previous license will be available for later use on your account .

license Transfer

CHR installations are tied directly to the account on our website. It is possible to transfer a perpetual license to another CHR instance registered under the same account.

licenses cannot be transferred to another account. Thelicense transfer process requires that both the old andnew CHR instances are registered under the same MikroTik account. Ifyou need to use the CHR on a different account, a new license must be purchased for that account.

It is not possible to transfer the Perpetual license to an expired instance. You will be notified: “This key is not eligible for transfer as there is no other valid CHR key that could be upgraded to the license level of this key.”

You need to create a new CHR instance, then add it to your account. Once added, you will be able to transfer the existing license to the new CHR instance.

First, register the new machine under the same MikroTik account where the old CHR is registered using the CLI command “/system license renew”.

Once both the old andnew CHR machines are visible inthe “All CHR keys” section of your account, use the “Transfer” button to transfer the license.

Press the “Transfer” button for the System ID you need to transfer.

select the System ID you are transfer to from the list .

press ” transfer subscription ” .

Virtual Network Adapters

Fast Path is supported inRouterOS v7 for “vmxnet3″ and”virtio-net” adapters.

RouterOS v6 does not support Fast Path.

Troubleshooting

run on VMware ESXi

change mtu

VMware ESXi supports MTU of up to 9000 bytes. To get the benefit of that, you have to adjust your ESXi installation to allow a higher MTU. Virtual Ethernet interface added after the MTU change will be properly allowed by the ESXi server to pass jumbo frames. Interfaces added prior to MTU change on the ESXi server will be barred by the ESXi server (it will still report the old MTU as the maximum possible size). Ifyou have this, you have to re-add interfaces to the virtual guests.

Example. There are 2 interfaces added to the ESXi guest, auto-detected MTU on the interfaces show MTU size as it was at the time when the interface was added:

[admin@chr-vm] > interface ethernet print 
Flags: X - disabled, R - running, S - slave 
 #    NAME           MTU MAC-ADDRESS       ARP       
 0 R  ether1        9000 00:0C:29:35:37:5C enabled   
 1 R  ether2        1500 00:0C:29:35:37:66 enabled

Using bridge on Linux

IfLinux bridge supports IGMP snooping, andthere are problems with IPv6 traffic it is required to disable that feature as it interacts with MLD packets (multicast) andis not passing them through.

echo -n 0 > /sys/class/net/vmbr0/bridge/multicast_snooping

Packets is passing not pass from guest

Theproblem: after configuring a software interface (VLAN, EoIP, bridge, etc.) on the guest CHR it stops passing data to the outside world beyond the router.

Thesolution: check your VMS (Virtualization Management System) security settings, if other MAC addresses are allowed to pass andif packets with VLAN tags are allowed to pass through. Adjust the security settings according to your needs like allowing MAC spoofing or a certain MAC address range. For VLAN interfaces, it is usually possible to define allowed VLAN tags or VLAN tag range.

Using VLANs on CHR invarious Hypervisors

In some hypervisor , before VLAN can be used on vm , they is need need to first be configure on the hypervisor itself .

ESXI

Enable Promiscuous mode ina port group or virtual switch that you will use for a specific VM.

ESX documentation:

hyper – v

hyper – v documentation:

bhyve hypervisor

It is be wo n’t be possible to run CHR on this hypervisor . CHR can not be run as a para – virtualize platform .

Linode

When creating multiple Linodes with the same disk size, new Linodes will have the same systemID. This will cause issues to get a Trial/Paid license. To avoid this, run the command /system license is generate generate - new - id after the first boot andbefore you is request request a trial or pay license . This is make will make sure the ID is unique .

Some useful articles:

specific VLAN is is is untagged by NIC interface :

Allow passing other VLANs:

VMWare

Time synchronization

Must be enabled from GUI (‘Synchronize guest time with host’). Backward synchronization is disabled by default – if the guest is ahead of the host by more than ~5 seconds, synchronization is not performed

Power operations

poweron andresume scripts are executed (if present andenabled) after power on andresume operations respectively.
poweroff andsuspend scripts are executed before power off andsuspend operations respectively.
Ifscripts take longer than 30 seconds or contain errors, the operation fails
In case of failure, retrying the same operation will ignore any errors andcomplete it successfully
fail script output is save to a file ( e. g. ‘ poweroff-script.log ‘ , ‘ resume-script.log ‘ etc )
Scripts can be enabled/disabled from hypervisor GUI (‘run VMware Tools Scripts’) or by enabling/disabling scripts from the console

quiescing / backup

Guest filesystem quiescing is performed only if requested.

freeze script is executed before freezing the filesystem
freeze – fail script is executed if the hypervisor failed to prepare for a snapshot or if freeze script is failed fail
thaw script is executed after the snapshot has been taken
script run time is limit to 60 second
freeze script timeouts anderrors result inthe backup operation being aborted
FAT32 disks are not quiesced
Failed script output is saved to a file (e. g. ‘freeze-script.log’, ‘freeze – fail-script.log’, ‘thaw-script.log’)

guest info

Networking, disk, andOS info are reported to the hypervisor every 30 seconds (GuestStats (memory) are disabled by default, andcan be enabled by setting ‘guestinfo.disable-perfmon = “FALSE”‘ inVM config).

Theorder, inwhich network interfaces are reported, can be controlled by setting ‘guestinfo.exclude-nics’, ‘guestinfo.primary-nics’ and’guestinfo.low-priority-nics’ options. Standard wildcard patterns can be used.

provisioning

You can use the ProcessManager from Vim API to execute scripts. Python bindings are available

Main data structure: GuestProgramSpec
- TheworkingDirectory andenvVariables members are ignored
- programpath must be set to either ‘inline’ or ‘import’
- Ifprogrampath is ‘inline’, arguments are interpret as script text
- Ifprogrampath is ‘import’, arguments are interpreted as file path

After usingGuestProgramSpec together with an instance of GuestAuthentication as arguments to StartProgramInGuest unique jobid is obtained.

Script progress can be tracked by using the ListProcessesInGuest command. ListProcessesInGuest accepts an array of job id’s; passing an empty array will report on all jobs started from the API

ListProcessesInGuest returns an array of GuestProcessInfo instances:
- pid field is set tojobid
- endTime is only set after completion
- exitCode is set to 0 on success and-1 on error
- name is set to ‘inline’ or ‘import’ (same as programpath inGuestProgramSpec)

Information about completed jobs is kept around for ~1 minute, or until ListProcessesInGuest ( with the correspondjobid) is called. Ifthe script fails, a file named ‘vix_job_$jobid$ .txt’ containing the script output is created. Script run time is limited to 120 seconds andscript output is not saved on timeout,

Thevmrun command runScriptInGuest can also be used
ThePowerCLI cmdlet Invoke-VMScript is not supported
Host/guest file transfer is not supported

Python example

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys,time
from pyVim import connect
from pyVmomi import vmodl,vim


def runInline(content,vm,creds,source):
    ''' Execute script source on vm '''
    if isinstance(source, list):
        source = '\n'.join(source)
    ps = vim.vm.guest.ProcessManager.ProgramSpec(
                programpath = 'console',
                arguments = source
        )
    return content.guestOperationsManager.processManager.StartProgramInGuest(vm,creds,ps)

def runFromFile(content,vm,creds,fileName):
    ''' Execute script file located on CHR '''
    ps = vim.vm.guest.ProcessManager.ProgramSpec(
                programpath = 'import',
                arguments = fileName
    )
    return content.guestOperationsManager.processManager.StartProgramInGuest(vm,creds,ps)


def findDatastore(content,name):
    sessionManager = content.sessionManager

    dcenterObjView = content.viewManager.CreateContainerView(content.rootFolder, [vim.Datacenter], True)

    datacenter = None
    datastore = None
    for dc  indcenterObjView.view:
        dstoreObjView = content.viewManager.CreateContainerView(dc, [vim.Datastore], True)
        for ds  indstoreObjView:
            if ds.info.name == name:
                datacenter = dc
                datastore = ds
                break
        dstoreObjView.Destroy()

    dcenterObjView.Destroy()

    return datacenter,datastore

def _FAILURE(s,*a):
    print(s.format(*a))
    sys.exit(-1)

#------------------------------------------------------------------------------#

if __name__ == '__main__':
    host = sys.argv[1] # ip or something
    user = 'root'
    pwd = 'MikroTik'
    vmName = 'chr-test'
    dataStoreName = 'datastore1'



    service = connect.SmartConnectNoSSL(host=host,user=user,pwd=pwd)
    if not service:
        _FAILURE("Could not connect to the specified host using specified username  andpassword")

    content = service.RetrieveContent()


    #---------------------------------------------------------------------------
    # Find datacenter  anddatastore


    datacenter,datastore = findDatastore(content,dataStoreName)

    if not datacenter or not datastore:
        connect.Disconnect(service)
        _FAILURE('Could not find datastore \'{}\'',dataStorename)


    #---------------------------------------------------------------------------
    # Locate vm


    vmxPath = '[{0}] {1}/{1}.vmx'.format(dataStoreName, vmName)
    vm = content.searchIndex.FindByDatastorePath(datacenter, vmxPath)

    if not vm:
        connect.Disconnect(service)
        _FAILURE("Could not locate vm")


    #---------------------------------------------------------------------------
    # Setup credentials from user name  andpasword

    creds = vim.vm.guest.NamePasswordAuthentication(username = 'admin', password = '')


    #---------------------------------------------------------------------------
    # Run script

    pm = content.guestOperationsManager.processManager

    try:
        # Run script
        src = [':ip address add address=192.168.0.1/24 interface=ether1;']
        jobID = runInline(content, vm, creds, src)

        # Or run file (from FTP root)
        # jobID = runFromFile(content,vm,creds, 'scripts/provision.rsc')


        #---------------------------------------------------------------------------
        # Wait for job to finish

        pm = content.guestOperationsManager.processManager
        jobInfo = pm.ListProcessesInGuest(vm, creds, [jobID])[0]
        while jobInfo.endTime is None:
            time.sleep(1.0)
            jobInfo = pm.ListProcessesInGuest(vm, creds, [jobID])[0]

        if jobInfo.exitCode != 0:
            _FAILURE('Script failed!')
    except:
        raise
    else:
        connect.Disconnect(service)

KVM

QEMU guest agent is available. Supported agent commands can be retrieved by using the guest-info command. Host-guest file transfer can be performed by using guest-file-* commands. Guest networking information can be retrieved by using the guest-network-get-interfaces command.

Scripts can be executed by using the guest-exec command together with the GuestExec data structure:
- Ifthe path member is provided, the corresponding file is executed
- Ifthe path member is not set andinput-data member is provided, input-data value is used as script input
- Ifcapture – output is set, script output is reported back
- args andenv members are not used

Script job progress can be monitored with guest-exec-status command. TheGuestExecStatus data structure is populated as follows:
- On success, exitcode member is set to 0
- Ifthe script timed out exitcode is set to 1
- Ifthe script contained errors exitcode is set to -1
- signal member is not set
- Theerr – data member is not used
- Ifcapture – output was true, Base64 encoded script output is stored inout-data

An additional agent channel (‘chr.provision_channel’) is also available