Session 1:- Veeam Backup at Full Throttle (Luca, Tom)
How to optimize VBR
in term of design and implementation?
Default settings -
it is good but to increase performance it is better to fine tune
Example: car. You need to fine tune to give you better performance.
What you can tune?
- VBS
- Proxy
- Backup repository
- Physical & Hypervisor
vSphere : Look for
- Vcenter VCSA- appliance size & performance
- Windows version - SQL (Express- small( 1 core/1 vcpu)- not multicore, Local, Remote)
Check -> number
of concurrent connection created by Veeam
Vcenter has
connection limit - use vcenter client (Backup connection establish from Backup
Job- min 2.)
Per default ,
maximum is 500 connection. Can increase the limit but will incur additional
memory usage
SQL connection - SQL
Express ( 1 socket but multicore)
ESXi limit on NBD.
NFC is limited on resources, especially on 1GB network. Recommend use 10G
Network (Use esxi 5.0 update 2)
-increase number of
buffer from 2048 to 4096
Reduce buffer cache
flush interval from 30 seconds to 20 seconds
Veeam fine tuning
Default value for
small environment
Large env - fine
tune!
Snapshot per
datastore. Limit max amount of active VM snapshot per datastore to prevent it
from being overfilled with snapshot delta. More concurrent snapshot, will incur
additional space consumption
Default value is
4. Change it with
MaxSnapshotPerDataStore (REG_DWORD)
-Available since
build 7.0.0771 (V7 Patch 2)
Snapshot per vCenter - limit max amount of active VM snashot per vcenter when using BfSS
Default value = 8
(in v7), or 20(in v8)
-Available since
build 7.0.0771 (V7 Patch 2)
Snapshot per ESXi -
limit max amount of active VM snashot per ESXi
Remove the snapshot
If you have fast
storage, you can increase the limit
MaxConcurrentDeletingSnapshotsForCluster
(DWORD) - default 4
MaxConcurrentDeletingSnapshotsForHost
(DWORD)- default 2
Backup from Storage
Snapshot
- In v9, can configure limit on the snapshot of the VM.
- Direct NFS
Optimizing proxy
& repository
- Jumbo frames
- Use VMXNet3 for virtual proxies but not PVSSCSI
- Tune NICs for throughput
- Look out for hardware optimization that introduce latency
Proxy sizing
- 2GB per core is ideal
- Estimated throughput is 50MB/s for each core
- If individual backup seem slow: vddkPreReadBufferSize - default 4MB. Increase can improve full performance (8MB or 16 MB)
Proxy recommendation
- Rule of thumb - 30-50 VM per core
- 4 vcpu / 8GB is great
- Increase proxy when require
Repository Sizing
- 4GB ram per core for large jobs
- Fast storage connectivity (local attach or 10GBE)
- Single server can be configured for multiple Veeam repository
Which type to
choose?
- Local disk / Nas/Dedupe
- Physical server + Fast Locak Disk (windows /linux)
- Fast caching RAID is the best, large stripe size (128K or greater) and SSD caching
Dedupe (great for
long term retention)
- support for DDBoast
- Support for Catalyst on HPstoreOnce
- Support ExaGrid
V9 - Check out : New per VM Backup Chain features.
Session 2:- Core
Platform Availability in Windows Server 2016 (Ben Armstrong)
Demo Video 1 -https://www.youtube.com/embed/UQbi210FAw4
Demo Video 2- https://www.youtube.com/embed/sUdOFQJKcHQ
Demo Video 3 -https://www.youtube.com/embed/Va5ER8nq3gs
Session 3:- Veeam General Session
New Product announcement :- Veeam Backup for Linux. - allow you to protect and recover Linux workload similar like Veeam Endpoint Backup. Since Linux do not has VSS, Veeam using it own built-in change tracking. Supported workload is Debian / Redhat. Currently it is under beta and it's FREE!. If you're interested to test it out, please register from here:- http://go.veeam.com/Linux
Customer also can leverage on existing Veeam Repositories as backup target.
Session 4:Backup
Repository Best Practices 2015 Edition (Anton)
Don't understand the
performance. Common issue - slow backup performance
5 Factors:
- Reliability
- Fast backup
- Fast restores
- DR from complete storage lost
- Lowest cost
Cost control
- Limit number of restore point on primary storage
- Using lower cost (per TB) secondary storage for long term retention
- At least 1 copy of backup
- Selective create offsite copies on tape in DR or in cloud
Backup repository -
Low end
- Any windows or linux server
- Physical server - local storage, DAS (JBOD) , SAN LUN
- Iscsi LUN via in guest iSCSI
- pDRM disk (require vsphere 5)
Deduplication
storage
- EMC data domain
- Exagrid
- HP StoreOnce
Top above brand, but
Veeam will work with any
VTL will be better
(faster) option for backup
Backup Repository to
Avoid
- Low end NAS and appliance (reliability)
- If stuck, use ISCSI instead of file protocol
- SMB (CIFS) network shares on Windows Server (reliability & performance)
- VMDK on VMFS (recoverability)
- Dependent on vSphere
- Windows Server 2012 Deduplication (scalability)
Immediate Future Support
- Windows Server 2016 deduplication
- REFS 2.0 - built on data corruption guard
RAW Disk (Internal,
DAS, SAN Storage)
- Raid 10 whenever you can (2x write penalty, but capacity suffer)
- RAID 5 - next economical choice (4x write penalty but greater risk
- RAID 6 - has most severe performance overhead( 6x write penalty)
Maximum performance
per spindle (lowest 7.2k, highest -15k SAS)
RAID volume
- RAID stripe size - typical i/o for veeam is 256KB to 512KB
- Windows Server 2012 default to 64KB. That's a lot of wasted I/O
- At least 128KB stripe size is highly recommended.
RAID array
- Fill as many drives as possible from the start to avoid expansion
- Low end storage has significant performance degradation when expanding.
File System:
- NTFS - recommend 64KB block size
- 16TB max file size limit before Win 2012 (which increase it to 256 TB)
- Format with /L to enable larger file record (enable large backup files on very large volumes)
- Refs 2.0
Backup Job Settings
- Always a performance vs disk space trade off
- Reverse incremental backup mode is 3x I/ per block
- Consider forever forward incremental backup mode instead
- Evaluate transform performance
Repository Load
- Limit concurrent jobs to a reasonable amount
- -use ingest rate throttling for cross SAN backup
Deduplication
Storage
Gain
- True global dedupe
- Lowest cost per TB
Pain
- Random access performance
- Poor ingest rates (bad as primary target)
- Require significant upfront investment
Best practice as
primary storage
- Do not use deduplication storage as primary backup repository
But if you have to
- Leverage on vendor specific integration
- Use backup modes without full backup transformation)
- Use active full instead synthetic full (if possible)
- If backup performance is still bad, consider VTL
- Use 16TB backup storage optimization for 4MB blocks
- Parallel processing may impact dedupe ration
- Limit to 1 task, instead of disabling
Best practice as
secondary storage
- Leverage on vendor specific integration
- No integration with your storage choice?
- Test backup copy retention processing performance. If too slow, consider using ACTIVE FUL OF BACKUP Copy Jobs
- Too late, already invested?
- Use as primary storage and leverage native replication
Best practice for
Backup Job settings
- Built in deduplication
- Keep on for best performance (except lowest end devices)
- Compression
- Instead of disable , keep optimal compression enabled in job and use decompress before storing repository action
- Dedupe friendly compression is not so friendly anymore
Why I love Tape?
- Cheaper
- Reliable
- Read only
- Customer success !
Tape is not dead!
What can go wrong
with disk?
- Storage level
corruption
- RAID controller ae your worst enemies
- Firmware and software bug are common
-Ransomware
-Unhappy Insider -
unhappy staff
Part of 3-2-1 Rule
-3 copies - 2
different media ,1 of them is offsite
-2 mean completely
different storage types - disk & tape
Storage Based
Replication
Cons - replicate bad
data just as good as it replicated
Your backup remains
in a single fault domain (3-2-1 is not meet)
Backup Copy vs
Storage Based copy
Prons:-
- Break the data loop
- Validates all source data
- Include backup file health check with auto healing
- Efficient backup optimization
How to make tape out
of disk?
Low end:-
- Use rotated drives
Mid range-
- Keep an offsite copy off-prem (cloud)
High end:
- Use hardware based WORM solutions
Check out v9 :- Scale Out backup
Repository - v9 (virtualize backup repository)
That's all for today. Stay Tuned for Day 3
