I'm having issues with my Rapid Recovery replication for one machine.  It is failing with the error "Repository is full" - but neither repository is full.

The total size of the machine's recovery points is 800GB.  Its local repository is 1.11TB in size (343GB free).  The remote core has a repository that is 950GB in size and has no data stored in it yet. 

Specific symptoms:

• The machine is protected locally and has successful base image and perhaps other recovery points as well
• The machine is added to replication without use of seed drive (successfully done multiple times with other larger servers)
• Replication starts, but fails - usually within a minute or two, but sometimes longer
• Error: The repository is full. Adjust your retention policy or delete the old recovery point chains to free up space, or add a new storage location to the existing repository. Call to service method [Remote servername/address] POST failed: The repository is full. Adjust your retention policy or delete the old recovery point chains to free up space, or add a new storage location to the existing repository

Other details:

There is a 100Mbps connection between the sites, so we usually replicate without seed drives and it takes approximately 24 hours.  I have it setup to have one repository per machine, and other larger machines are successfully replicating with no issue.

Things I've tried:

• Giving the remote repository more space (at one point it was larger than the local repository)
• Giving the local repository more space
• Recreating both the local and remote repositories using new names
• Removing and re-protecting the machine using the local core
• Removing/recreating the replication

I'm out of ideas.  I have it giving it another try now and it's working on replication and has been for about 30 minutes, but just earlier today it worked on replication for about 50 minutes before reporting the repository was full.

Rapid Recovery Version: 6.0.2.144

Hi all,

I'm hoping I can get some clarification on the encryption in Appassure.

We have agents in remote offices backing up to onsite cores and then those cores replicating offsite to our central office. Those replication jobs are traveling over vpn's setup from each office to the central office.

We have several small offices and micro offices that we don't have dedicated vpn's and would prefer to keep it that way. Of course we need to be sure that data traveling over the internet is encrypted. My understanding is the data in Appassure is encrypted on the core and therefore when it is traveling offsite it remains encrypted.

So if we replicated offsite from a remote core to our central office without a vpn the data is safe and completely encrypted during travel. Is that the case?

Additionally, my understanding is the data traveling from the agent to the local core is not encrypted and travels over the lan in "clear text" for lack of a better description. Is that the case?

Thanks for any information you can provide

I tried to do a big restore (300GB folder) & many of the files simply 'failed to copy'.  Since you can't see the entire file path during the 'restore' (which is just a copy procedure), I have no idea how to figure out why they failed.  There were so many I finally got tired of hitting 'ignore' and stopped the copy

Is there a log created during a 'restore'?

Thx

Hi Guys

Is there a way to setup alerting in rapid recovery, to get an alert if a backup failed?

Would be very efficient instead of going into each server.

Thanksr

We have a DL4000 with several TBs of data and many agents, this is used as a replication target only.

The inbuilt rollup job pauses all incoming replications so we stopped using that. We use the rollup script here: support.software.dell.com/.../205880

My issue is, when its going a deletion job, the script sticks and wont rollup any agents. My reading of the KB suggested it would move onto the next agent, which its not doing, or maybe the delete would affect any agent rollup anyway.. either way.

Can you pause deletion jobs via powershell? if so, i can stick a pause in at the start and a resume at the end. Any other suggestions maybe?

We use N-Able to monitor any errors. In AppAssure it all worked perfectly, however now the upgrade to RR, the event log name changed (which is easy enough to change in the script) however all the event IDs logged by RR are different too. This is a big issue as we dont know all the IDs for warnings and errors.

Does anyone know or have a list of the IDs generated by RR ?

is it possible to restore from an archive when booting the CD created with Appassure GUI, if the archive is attached to the USB drive on a FN430?

i can`t seem to get the drive to show up for some reason.

So this may be a DL4000 only issue however..

Upgraded from 5.4.3 to 6.0.2.

Events stopped working. There is no DELL event log (ticked to log in event log in settings) and all the ticks by the notifications you want were unticked. ticking simply untick themselves.

Core has been rebooted. Dell support say they will come back to me by Tuesday, thats too long..

I also exported the event reg key from working core (not DL4000) but they looked the same anyway and RR still doesnt see any alerts selected.

Anyone have any ideas or come up against this?

During the install. Does anyone have any ideas why?

The install\rollback also created another service, which isnt started but runs same exe, as below

"AppAssure AppAssure 5 Core Service"

"Manages repositories, authentication for protected machines, schedules for transferring data for backup and replication, exporting to VMs, reporting, and restoring data."

Path: "C:\Program Files\AppRecovery\Core\CoreService\Core.Service.exe"

why are updates not getting download or notifying the server that there are update anymore?  Since updating to RR6 it never tells me there is any updates.

I have to manually check the rapid recovery website in order to find out what is the current release.

this always worked with appassure. 

Hi,

As the RapidRecovery agent kernel module appears to be closed-source, we can't build our own modules for custom Linux builds.

We've been looking at building more of our infrastructure on Docker, and one thing we'd like to be able to do is back up storage volumes on our Docker hosts.

CoreOS is a distro built for running Docker at high performance. There's an article on building kernel modules for it here.

It would be great if you could enable us to use the two together. Ideally by giving us access to the kernel module source so that we can compile a module ourselves. Or otherwise by creating a Docker registry containing containers that have the kernel module built in.

Thanks!

All, I would like to request an alert to know when base images start.  We need to know to control the replication of those, otherwise we get calls from clients because bandwidth is maxed.

All, I would like there to be some checks in place to confirm the integrity of a back up after an OS is not cleanly shutdown to avoid new base images all the time.

All, has anyone else upgraded to RR from 5.4.3 and notice that the core service is not starting when the core is rebooted?  I called support and they wanted to run repairs, remove patches, reinstall etc.  I thought the service was just timing out so I looked for 7000 and 7011 in the app log but couldnt find them.  This is happening on over half the cores we upgraded so before I went through the who deal of repairing, removing patches, installing patches etc, wanted to start here.

support.microsoft.com/.../922918 

Hi all,

I'm looking for some information on how to handle recovery point check failures. We see them rarely but when we do we are not sure exactly what steps we should take.

When this error comes up does that mean just the most recent snapshot is bad or is the entire chain right back to the base image bad? Will the next snapshot fix the issue and we just have to make a note that on this particular date the snapshot is bad? Does appassure mark that snapshot as bad?

Its rarely the data partitions that are bad like C: or D:, its almost always the EFI partition, the recovery partition or the system volume. In our case we don't plan on booting these images, just restoring data so do we even have to worry about those?

Any guidance is appreciated.

i`ve a problem with the backup due to transfer timeout (Timeout getting the VSS writers from 'yyyy') - 10min is not enough (vssadmin list writers  need about ~25min) .

When I try to increase transfer timeout to 30min and save the transfer settings (AA->machine->configuration->transfer settings) get a timeout error :

"Error

Call to service method localhost:8006/.../config PUT failed: Service method call timed out after 00:05:00.0129505"

Has other experienced this problem and do you have any suggestions? 

Server 2012 R2 STD   

SQL 2008 Express

2 instances one with two databases

The transfer failed: 'There was a problem with the VSS subsystem on 'server2012'.'
The VSS writer 'SqlServerWriter' failed during the 'DoSnapshotSet' phase with error WriterErrorNonRetryable. The VSS writer state is FailedAtPrepareSnapshot

Exception chain:

The VSS writer 'SqlServerWriter' failed during the 'DoSnapshotSet' phase with error WriterErrorNonRetryable. The VSS writer state is FailedAtPrepareSnapshot
There was a problem with the VSS subsystem on 'server2012'.
The transfer failed: 'There was a problem with the VSS subsystem on 'server2012'.'
One or more errors occurred.

STACK Trace:

Server side:

Replay.Agent.Contracts.Transfer.TransferFailedException: The transfer failed: 'There was a problem with the VSS subsystem on 'server2012'.' ---> System.AggregateException: One or more errors occurred. ---> Replay.Core.Contracts.Transfer.VssGeneralErrorException: There was a problem with the VSS subsystem on 'server2012'. ---> Replay.Agent.Contracts.ShadowCopy.ShadowCopyWriterFailedException: The VSS writer 'SqlServerWriter' failed during the 'DoSnapshotSet' phase with error WriterErrorNonRetryable. The VSS writer state is FailedAtPrepareSnapshot ---> Replay.Common.Contracts.ReplayException


--- End of inner exception stack trace ---
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.GatherAndReportWriterStatus(String phase, Boolean throwOnError)
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.PreBackup()
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.TakeSnapshot()
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.TakeSnapshot(TakeSnapshotRequest request, VolumeName[] volumesToSnap, IVssImplementation vss, IWin32 win32, ITevoLibWrapper tevoLibWrapper, IWmiWrapper wmiWrapper, IStorageMetadata storageMetadata, Dictionary2& shadowCopyFiles)
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyService.TakeSnapshot(TakeSnapshotRequest request, VolumeName[] volumesToSnap, IStorageMetadata storageMetadata, Dictionary2& shadowCopyFiles)
   at Replay.Agent.Implementation.Transfer.TransferSessionBase.PrepareShadowCopy(TakeSnapshotRequest request, IDriversService driversService, ITevoLibWrapper driverApi, IShadowCopyService shadowCopyService, IStorageMetadata storageMetadata, Dictionary2& shadowCopyFiles, List1& volumesToAssumeAsNotSnappable)
   at Replay.Agent.Implementation.Transfer.TransferSessionBase.PrepareToTakeSnapshot(TakeSnapshotRequest request, IShadowCopyService shadowCopyService, IDriversService driversService, ITevoLibWrapper driverApi, IStorageMetadata metadata, Dictionary2& errorVolumes, IShadowCopy& shadowCopy, Dictionary2& shadowCopyFiles, List`1& volumesToAssumeAsNotSnap)
   at Replay.Agent.Implementation.Transfer.TransferSession.StartTransferSession(String agentHostName, IPAddress coreIPAddress, TakeSnapshotRequest request, CancellationToken cancellationToken, IShadowCopyService shadowCopyService, IDriversService driversService, IStorageService storageService, ITransferDataServerFactory transferDataServerFactory, ITevoLibWrapper driverApi, AgentTransferSettings transferSettings)


--- End of inner exception stack trace ---
   at Replay.Core.Implementation.Transfer.TransferJob.TakeSnapshotInternal(TransferGroupItem transferGroupItem)
   at Replay.Core.Implementation.Transfer.TransferJob.<>cDisplayClass8c.b8a(TransferGroupItem item)


--- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Replay.Core.Implementation.Transfer.TransferJob.ExecuteParallel(Action`1 action)
   at Replay.Core.Implementation.Transfer.TransferJob.TakeSnapshot()
   at Replay.Core.Implementation.Transfer.TransferJob.TransferTaskInternal()
   at Replay.Core.Implementation.Transfer.TransferJob.TransferAgentTask()
   at Replay.Core.Implementation.Transfer.TransferJob.TransferTask()
   at System.Threading.Tasks.Task.Execute()


Replay.Agent.Contracts.ShadowCopy.ShadowCopyWriterFailedException: The VSS writer 'SqlServerWriter' failed during the 'DoSnapshotSet' phase with error WriterErrorNonRetryable. The VSS writer state is FailedAtPrepareSnapshot ---> Replay.Common.Contracts.ReplayException


--- End of inner exception stack trace ---
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.GatherAndReportWriterStatus(String phase, Boolean throwOnError)
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.PreBackup()
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.TakeSnapshot()
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyInstance.TakeSnapshot(TakeSnapshotRequest request, VolumeName[] volumesToSnap, IVssImplementation vss, IWin32 win32, ITevoLibWrapper tevoLibWrapper, IWmiWrapper wmiWrapper, IStorageMetadata storageMetadata, Dictionary2& shadowCopyFiles)
   at Replay.Agent.Implementation.ShadowCopy.ShadowCopyService.TakeSnapshot(TakeSnapshotRequest request, VolumeName[] volumesToSnap, IStorageMetadata storageMetadata, Dictionary2& shadowCopyFiles)
   at Replay.Agent.Implementation.Transfer.TransferSessionBase.PrepareShadowCopy(TakeSnapshotRequest request, IDriversService driversService, ITevoLibWrapper driverApi, IShadowCopyService shadowCopyService, IStorageMetadata storageMetadata, Dictionary2& shadowCopyFiles, List1& volumesToAssumeAsNotSnappable)
   at Replay.Agent.Implementation.Transfer.TransferSessionBase.PrepareToTakeSnapshot(TakeSnapshotRequest request, IShadowCopyService shadowCopyService, IDriversService driversService, ITevoLibWrapper driverApi, IStorageMetadata metadata, Dictionary2& errorVolumes, IShadowCopy& shadowCopy, Dictionary2& shadowCopyFiles, List`1& volumesToAssumeAsNotSnap)
   at Replay.Agent.Implementation.Transfer.TransferSession.StartTransferSession(String agentHostName, IPAddress coreIPAddress, TakeSnapshotRequest request, CancellationToken cancellationToken, IShadowCopyService shadowCopyService, IDriversService driversService, IStorageService storageService, ITransferDataServerFactory transferDataServerFactory, ITevoLibWrapper driverApi, AgentTransferSettings transferSettings)

Hi.

We upgraded appassure from 5.4.3 to rapidrecovery 6.0.2 in june. We had a problem with notification and we find the KB to resolve the problem. All was fine until last week. Our server restart due to a WIndows update. Now, rapidrecovery can't send any kind of notification (by email, toaster, windows event...) . I tried to apply the KB190334 without success.

Here a sample of the core log file:

ERROR 2016-08-17T08:11:26 [3058] - Replay.Core.Implementation.Events.Notificators.AlertsNotificationService ()
Failed to refresh notificator of type Replay.Core.Implementation.Events.Notificators.EmailNotificator
System.NullReferenceException depth 0: Object reference not set to an instance of an object. (0x80004003)
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetSingleNotificationGroupSettings(IEnumerable`1 settingsGroups) +0x5 6000e13
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.<GetAgentsSettings>b__2(KeyValuePair`2 agent) +0x0 6000e17
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) +0x49 6000431
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector) +0x0 6000430
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetAgentsSettings(IDictionary`2 agentsSettings) +0x2a 6000e01
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.RefreshNotificationSettings(IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x8 6000e0a
at Replay.Core.Implementation.Events.Notificators.AlertsNotificationService.RefreshNotificator(IAlertsNotificator notificator, CoreAlertsSettings generalAlertsSettings, IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x0 60036f6

ERROR 2016-08-17T08:11:26 [3058] - Replay.Core.Implementation.Events.Notificators.AlertsNotificationService ()
Failed to refresh notificator of type Replay.Core.Implementation.Events.Notificators.WindowsEventLogNotificator
System.NullReferenceException depth 0: Object reference not set to an instance of an object. (0x80004003)
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetSingleNotificationGroupSettings(IEnumerable`1 settingsGroups) +0x5 6000e13
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.<GetAgentsSettings>b__2(KeyValuePair`2 agent) +0x0 6000e17
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) +0x49 6000431
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector) +0x0 6000430
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetAgentsSettings(IDictionary`2 agentsSettings) +0x2a 6000e01
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.RefreshNotificationSettings(IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x8 6000e0a
at Replay.Core.Implementation.Events.Notificators.AlertsNotificationService.RefreshNotificator(IAlertsNotificator notificator, CoreAlertsSettings generalAlertsSettings, IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x0 60036f6

ERROR 2016-08-17T08:11:26 [3058] - Replay.Core.Implementation.Events.Notificators.AlertsNotificationService ()
Failed to refresh notificator of type Replay.Core.Implementation.Events.Notificators.SyslogNotificator
System.NullReferenceException depth 0: Object reference not set to an instance of an object. (0x80004003)
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetSingleNotificationGroupSettings(IEnumerable`1 settingsGroups) +0x5 6000e13
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.<GetAgentsSettings>b__2(KeyValuePair`2 agent) +0x0 6000e17
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) +0x49 6000431
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector) +0x0 6000430
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetAgentsSettings(IDictionary`2 agentsSettings) +0x2a 6000e01
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.RefreshNotificationSettings(IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x8 6000e0a
at Replay.Core.Implementation.Events.Notificators.AlertsNotificationService.RefreshNotificator(IAlertsNotificator notificator, CoreAlertsSettings generalAlertsSettings, IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x0 60036f6

ERROR 2016-08-17T08:11:26 [3058] - Replay.Core.Implementation.Events.Notificators.AlertsNotificationService ()
Failed to refresh notificator of type Replay.Core.Implementation.Events.Notificators.SnmpNotificator
System.NullReferenceException depth 0: Object reference not set to an instance of an object. (0x80004003)
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetSingleNotificationGroupSettings(IEnumerable`1 settingsGroups) +0x5 6000e13
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.<GetAgentsSettings>b__2(KeyValuePair`2 agent) +0x0 6000e17
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) +0x49 6000431
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector) +0x0 6000430
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.GetAgentsSettings(IDictionary`2 agentsSettings) +0x2a 6000e01
at Replay.Core.Implementation.Events.Notificators.AlertsNotificatorBase`1.RefreshNotificationSettings(IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x8 6000e0a
at Replay.Core.Implementation.Events.Notificators.AlertsNotificationService.RefreshNotificator(IAlertsNotificator notificator, CoreAlertsSettings generalAlertsSettings, IEnumerable`1 coreSettings, IDictionary`2 agentsSettings) +0x0 60036f6

Can someone help me.

Thanks.

I need to make a seed drive for a few VMs.  but I don't need all the incrementals.   Is it safe to keep onlythe Baseand the most recent Incremental, and delete all the incrementals between?  I think this will help reduce size of the seed, and speed things up. 

I think the answer is yes.

We were running appassure 5.4 and were recently upgraded to 6.0.2.144.

Now that we are on this new version we are unable to run attachability checks on a couple servers.

We are told from our install tech that this new version does not support sql express db's.  He also said that dell told him that that attachability checks are a per sever option.  Pretty much telling us that since we have both sql and sql express db's that none of them will run.  

I also noticed that there is a warning in the details of this particular server stating :
Information about the following SQL instances is not available:

• RWS

  Failed to connect to the Sql Server using all possible protocols. Check that at least one protocol is enabled

Not sure why this is happening but it is showing on all of our sql express 2008 dbs.

I was hoping someone could explain if our tech is right or if something else is going on here.

Thanks