Microsoft Sync Framework

Microsoft Sync Framework is a data synchronization platform from Microsoft that can be used to synchronize data across multiple data stores. Sync Framework includes a transport-agnostic architecture, into which data store-specific synchronization providers, modelled on the ADO.NET data provider API, can be plugged in. Sync Framework can be used for offline access to data, by working against a cached set of data and submitting the changes to a master database in a batch, as well as to synchronize changes to a data source across all consumers (publish/subscribe sync) and peer-to-peer synchronization of multiple data sources. Sync Framework features built-in capabilities for conflict detection – whether data to be changed has already been updated – and can flag them for manual inspection or use defined policies to try to resolve the conflict. Sync Services includes an embedded SQL Server Compact database to store metadata about the synchronization relationships as well as about each sync attempt. The Sync Framework API is surfaced both in managed code, for use with .NET Framework applications, as well as unmanaged code, for use with COM applications. It was scheduled to ship with Visual Studio 2008 in late November 2007.^[1]

Architecture

The Sync Framework runtime provides synchronization functionality, without being tied to any data store or data transport protocols. By providing data source specific synchronization providers, any data source can be supported. For example, using proper synchronization providers, files can be synchronized across computers, project updates synchronized across project participants, or media synchronized across devices. Sync Framework ships with three providers: Microsoft Sync Services for ADO.NET, Sync Services for File Systems, and Sync Services for SSE. Sync Services can be used to synchronize devices by supplying providers for the device. Similarly, PIM software such as Microsoft Office Outlook and media libraries such as Windows Media Player can also be supported by providing suitable providers.

The providers are used to enumerate the items in a data store, each identified by an Item ID. In addition, they also have to maintain synchronization metadata and the state of the data store, so that changes can be enumerated quickly. The metadata is maintained for every instance of the data store (replica) that the provider is attached to. The metadata maintained includes the replica ID, tick count (representing progression in time), conflict log, tombstone log, and the set of the changes the data store has seen (knowledge). A replica ID and tick count pair makes up a version and encodes the state of the data store until that time. Sync Framework defines a set of operation for the Knowledge object for a replica: Contains which determines if the store contains a specified change, Union to merge two knowledge sets, Project to project out the knowledge for a subset of the items, and Exclude to create a new knowledge set without the changes for a subset of the items. The metadata is managed by the metadata storage service which uses an in-process SQL Server Compact database to store the metadata on a per-provider basis.

The Sync Services API operates by creating a synchronization session, represented by a Session object. A synchronization session synchronizes data across two synchronization providers - one for the source data store and the other for the destination. Instances of both the providers are passed to the Session object. During a synchronization session, the destination provider sends the knowledge set of the store. The source provider compares the knowledge of the destination with the change set in the source to enumerate the changes and then transfer it to the destination. The destination provider makes sure the changes are not conflicting and merges the changes and updates the knowledge.

Snapshot sync (download-only sync): The data in the data source (or a subset of it) is synchronized with clients.
Upload-only sync: Data in the client is merged to the source replica.
Bidirectional sync: Both the data sources can be modified independently and changes are synchronized with each other. An n-level sync is achieved by performing multiple bidirectional synchronizations.

Sync Services for ADO.NET

Microsoft Sync Services for ADO.NET is the synchronization provider for synchronizing across databases using ADO.NET. ADO.NET Datasets are synchronized between the source and the destination, which are then persisted to a database server. It can also support data sources other than a relational database, like an XML database or web service as long as a proxy is provided to abstract the data source and a data provider is available for the proxy.

The Sync Services for ADO.NET provider is intended for use in offline applications, where data from the central database is cached locally. The application works against the cached data, and the changes are uploaded in a batch. In addition, the provider can also be used for collaborative applications, where each application will work against its local dataset, which will be synchronized periodically in a peer-to-peer manner with the other participants. Locally, the datasets can be stored either by using the SQL Server Compact database or any other database server supporting ADO.NET. Sync Services for ADO.NET allows incremental change tracking, which allows only the changes to be replicated rather than replicating the entire copy.

Sync Services for File Systems

The Sync Services for File Systems provider is used to synchronize two file system locations, which can either be local folders or network shares. In addition to mirroring new files, changes to existing files are also synchronized. Changes to files are detected by using timestamps, or optionally, by hashing the file contents. Conflicting changes to the same file are detected and can be set to be automatically resolved. For conflicting updates to a same file, the newer edit will be kept. If a file is deleted in one replica but updated in another, the update will take precedence over the delete. If two files with different content are created with the same name across two replicas, during the sync operation, the one created later will be persisted. If a rename operation caused the files to get the same name, both are retained by renaming one of them. Any deletes can be configured to move the file to the Recycle Bin, so that it can be recovered if necessary. The Sync Services for File Systems provider also provides a preview mode which enumerates the actions that will be taken for a sync operation, without actually performing the operations, with a view to letting the users review the changes that will be made. The synchronization is performed in a peer-to-peer manner. Neither Sync Framework or the Sync Services for File Systems provider perform any authentication before accessing the files; so any authentication is the job of the application using the Sync Framework API. The files are transferred without encryption. To use encryption in transit, custom providers that uses an encrypted TCP connection needs to be used. The Sync Services for File Systems provider also supports static filters to exclude files based on wildcards or attributes. In the first CTP release, however, the Sync Services for File Systems provider does not sync either NTFS security descriptors or Alternate Data Streams.

Sync Services for FeedSync

The Sync Services for FeedSync provider can be used to help synchronize replicas by creating a FeedSync enabled feed, either in RSS or ATOM formats, which can then be subscribed to by interested parties. The provider can also be used to extract items from a FeedSync feed and merge the changes back to the data store. Sync Services for FeedSync uses another provider to connect to the data store.

Sync Services for FeedSync provides services that can be used to help synchronize the data of a replica with RSS and Atom feeds. (A replica is a particular repository of information to be synchronized.) By using the FeedSync producer service, a synchronization application can work with a synchronization provider to create a list of items from a replica and put them in an RSS or Atom XML stream. These items can then be published to interested subscribers. Similarly, the FeedSync consumer service helps a synchronization application take an input RSS or Atom XML stream, extract items from it, and then use a synchronization provider to apply only the appropriate changes to a replica. Because Sync Framework underlies the exchange of feed items, two feeds can be cross-subscribed and easily synchronized with one another as peers in a synchronization community. (A synchronization community is a set of replicas that keep their data synchronized with each other.)

Licensing

Microsoft Sync Framework is free on Windows and Windows Mobile devices. Support for other platforms is available through commercial licensing and porting kits.^[2]