Veeam tape backup job fails because of Storage not found
Last week I had to troubleshoot a Veeam Backup&Replication tape backup job that failed because of storage objects were not found. This GFS job had to copy multiple backup jobs to tape, which usually worked fine.
I this environment, synthetic full and tape backup are scheduled at the same day. Full prior to tape job. Normally I do see any problems with this configuration. But last week this error occurred in the tape job:
12.01.2022 07:04:45 :: Error: Storage ('c97b8ddb-a9dc-4b49-b441-dc1031100a58', CreationTime '08.01.2022 23:15:25') not found
As a result the whole job failed.
After taking a look at the logs, I found out, the related storage object was a VIB-file (incremental backup file) of one of the backup jobs, tape job had to copy to tape.
[08.01.2022 23:45:07] <01> Info [CTapeVmTaskBuilder] Storage c97b8ddb-a9dc-4b49-b441-dc1031100a58, mediaset: Weekly, path: servername.vm-187205D2022-01-08T231525_8F3A.vib, creation time: 08.01.2022 23:15:25, is synthetic: True, is synthetic full: True
Although the disk backup job was running at time, the file was not there at all. Why? The reason is quite simple: For this day a synthetic full was scheduled. So Veeam creates a VIB-file, which should be replaced by a VBK-file (full backup file) at the time of merging. Therefore the error in the tape job: Tape job starts and searches for backup files to copy to tape. Here the restore point of the VIB-file was selected, because backup job has not performed the merge by this time. Later, when tape job wanted to access the VIB-file, it was replaced by the VBK-file – because of merging for synthetic full. So, storage file was not found. In my case, disk backup job needed a re-try, therefore the merge-process started much later than normally.
When you suffer from this error more often, you could try to create some time buffer between full and tape job. Another solution could be to configure tape job to use already existing restore points instead of waiting for new respectively currently running ones.