CN103188282B - Method, system and the equipment of file synchronization - Google Patents
Method, system and the equipment of file synchronization Download PDFInfo
- Publication number
- CN103188282B CN103188282B CN201110444163.5A CN201110444163A CN103188282B CN 103188282 B CN103188282 B CN 103188282B CN 201110444163 A CN201110444163 A CN 201110444163A CN 103188282 B CN103188282 B CN 103188282B
- Authority
- CN
- China
- Prior art keywords
- data
- file
- transmission
- purpose data
- file data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention provides the method for file synchronization, system and equipment, wherein, this method includes：Purpose data sources unit receives the file data of source server transmission, storage file data, and generates transmission daily record；Purpose data source copy cell obtains the corresponding document data stored in the filename of the file data received, copy purpose data sources unit from transmission daily record, is stored；Data processing unit carries out data processing from purpose data source copy cell copied files data.The present invention program can avoid read/write conflict during file synchronization.
Description
Technical field
The present invention relates to method, system and the equipment of data processing technique, more particularly to file synchronization.
Background technology
, it is necessary to which file data is transferred into destination server on a large scale after each source server generation file data, by
Destination server is focused on to file data, obtains the information needed.The file data can be specially user to net
Each record that browses for showing content on page, the displaying content is, for example, news information, sport information, advertising message on webpage
Deng.
Source server by file data transfer destination server, and destination server to the file data of write-in at
The process of reason, is the process of file synchronization.The process is described in detail below：Destination server connects including purpose data source
Receive unit and data processing unit；The file data transfer of generation is given purpose data by source server according to existing transmission mechanism
Source receiving unit, purpose data sources unit receives the file data of source server transmission, storage file data, and generates biography
Defeated daily record, the information such as filename of the transmission daily record comprising the file data write and point of corresponding write time；At data
Manage unit and read file data from purpose data sources unit, carry out data processing, and delete purpose data sources unit
The corresponding document data of middle storage.
In order to be carried out to the file data handled in purpose data sources unit with also untreated file data
Difference, data processing unit is read after file data from purpose data sources unit, just deletes purpose data sources unit
The corresponding document data of middle storage.According to existing transmission mechanism, in data transmission procedure, source server is by purpose data source
The data of the data stored in receiving unit and source server itself storage are compared, and are judged in purpose data sources unit
Whether All Files data in source server are stored, if it is not, then will not store in purpose data sources unit
Fractional transmission gives purpose data sources unit.Connect due to purpose data source will be deleted after data processing unit progress data processing
The corresponding document data stored in unit are received, this will cause source server repeatedly to write file to purpose data sources unit
Data, cause read/write conflict.
In order to be carried out to the file data handled in purpose data sources unit with also untreated file data
Difference, also can be by the way of being recorded in real time to the file data that data processing unit has been handled, but this mode
Workload is too big, and the data volume of record is very big, does not use typically, and deletes purpose data after the more data processing using progress
The mode of file data in the receiving unit of source, but there is the defect of read/write conflict in this mode.
The content of the invention
The invention provides a kind of method of file synchronization, this method can avoid read/write conflict during file synchronization.
The invention provides a kind of system of file synchronization, the system can avoid read/write conflict during file synchronization.
The invention provides a kind of equipment of file synchronization, the equipment can avoid read/write conflict during file synchronization.
A kind of method of file synchronization, this method includes：
Purpose data sources unit receives the file data of source server transmission, storage file data, and generates transmission
Daily record；
Purpose data source copy cell obtains the filename of the file data received from transmission daily record, copies purpose number
According to the corresponding document data stored in the receiving unit of source, stored；
Data processing unit reads file data from purpose data source copy cell, carries out data processing.
A kind of system of file synchronization, the system includes source server and destination server, and the destination server includes
Purpose data sources unit, purpose data source copy cell and data processing unit；
The source server, for transmitting file data to the purpose data sources unit；
The purpose data sources unit, the file data for receiving the source server transmission, storage file number
According to, and generate transmission daily record；
The purpose data source copy cell, the file for obtaining the file data received from the transmission daily record
The corresponding document data stored in name, copy purpose data sources unit, are stored；
The data processing unit, for reading file data from purpose data source copy cell, carries out data processing.
A kind of equipment of file synchronization, the equipment include purpose data Src StUnit, purpose data source copy cell and
Processing unit；
The purpose data sources unit, the file data for receiving source server transmission, storage file data, and
Generation transmission daily record；
The purpose data source copy cell, the file for obtaining the file data received from the transmission daily record
The corresponding document data stored in name, copy purpose data sources unit, are stored；
The data processing unit, for reading file data from purpose data source copy cell, carries out data processing.
From such scheme as can be seen that in the present invention, the file data that purpose data sources unit is received is copied to
In purpose data sources copy cell, data processing unit reads file data from purpose data source copy cell again, carries out
Processing.So, handled without reading file data from purpose data sources unit, it is not necessary to purpose data sources
The file data of unit storage is deleted, so that, it is to avoid read/write conflict during file synchronization.
Brief description of the drawings
Fig. 1 is the system structure diagram of file synchronization of the present invention；
Fig. 2 is the method indicative flowchart of file synchronization of the present invention.
Embodiment
For the object, technical solutions and advantages of the present invention are more clearly understood, with reference to embodiment and accompanying drawing, to this
Invention is further described.
In the present invention, the file data that purpose data sources unit is received is copied out, then the text to copying out
Number of packages evidence is read out and handled.So, handled, just need not without reading file data from purpose data sources unit
The file data that purpose data sources units is stored is deleted, so that, it is to avoid read/write conflict during file synchronization.
It is the system structure diagram of file synchronization of the present invention referring to Fig. 1, the system includes source server and purpose service
Device, the destination server includes purpose data sources unit, purpose data source copy cell and data processing unit；
The source server, for transmitting file data to the purpose data sources unit；
The purpose data sources unit, the file data for receiving the source server transmission, storage file number
According to, and generate transmission daily record；
The purpose data source copy cell, the file for obtaining the file data received from the transmission daily record
The corresponding document data stored in name, copy purpose data sources unit, are stored；
The data processing unit, for reading file data from purpose data source copy cell, carries out data processing.
In order to be carried out to the file data handled in purpose data source copy cell with also untreated file data
It may include to delete subelement in difference, the data processing unit, for being read from purpose data source copy cell after data, delete
Except the corresponding document data stored in purpose data source copy cell；So, it is stored in the text in purpose data source copy cell
Number of packages evidence, is all also untreated file data.
Purpose data sources unit receives the file data of source server transmission, is stored and generates transmission daily record；
One file has not been transmitted and finished just in transmitting procedure, if now purpose data source copy cell is copied to this document,
Because the data of this document are imperfect, follow-up result will be caused mistake occur；Therefore, the present invention connects to purpose data source
Receive unit and purpose data source copy cell sets lock mechanism, to avoid the defect.The lock mechanism is connect by purpose data source
Receiving the lock mark in unit sets inquiry subelement and copy subelement in subelement, and purpose data source copy cell real
It is existing；Specifically：
It is described lock mark set subelement, for generation transmission daily record when, judge file data whether end of transmission, such as
Fruit is that lock mark then is set into end of transmission, otherwise is set to transmit by lock mark；
The inquiry subelement, for judging the lock mark, if lock is designated end of transmission, to described
Copy subelement and send enabled instruction, transmitted if lock is designated, wait until that lock mark is set to end of transmission just to institute
State copy subelement and send enabled instruction；
The copy subelement, specifies for receiving described start, the text of end of transmission is obtained from transmission daily record
The corresponding document data stored in the filename of number of packages evidence, copy purpose data sources unit, are stored.
The file data can be specially user browses record to each displaying content on webpage, described to show content for example
For the news information on webpage, sport information, advertising message etc..
It is the method indicative flowchart of file synchronization of the present invention, it comprises the following steps referring to Fig. 2：
Step 201, the file data of purpose data sources unit reception source server transmission, storage file data, and
Generation transmission daily record.
File data is write purpose data sources unit, purpose data source by source server according to existing transmission mechanism
Receiving unit is received and storage file data, also generation transmission daily record, the file of file data of the transmission daily record comprising write-in
The information such as name and corresponding write time point.
Step 202, purpose data source copy cell obtains the filename of the file data received from transmission daily record, copies
The corresponding document data stored in shellfish purpose data sources unit, are stored.
Step 203, data processing unit reads file data from purpose data source copy cell, carries out data processing.
In order to be carried out to the file data handled in purpose data source copy cell with also untreated file data
Difference, data processing unit is read after file data from purpose data source copy cell, can delete purpose data source copy cell
The corresponding document data of middle storage.
Purpose data sources unit receives the file data of source server transmission, is stored and generates transmission daily record；
One file has not been transmitted and finished just in transmitting procedure, if now purpose data source copy cell is copied to this document,
Because the data of this document are imperfect, follow-up result will be caused mistake occur；Therefore, the present invention connects to purpose data source
Receive unit and purpose data source copy cell sets lock mechanism, to avoid the defect.The lock mechanism is specifically included：
In Fig. 2 flow, in step 201 purpose data sources unit storage file data and generate transmission daily record when,
Judge file data whether end of transmission, if it is, by lock mark be set to end of transmission, otherwise will lock mark be set to just
In transmission；Correspondingly, perform before step 202, this method also includes：Purpose data source copy cell is carried out to the lock mark
Judge, if lock is designated end of transmission, the file data of end of transmission is obtained in performing the daily record from transmission
The step of filename, be designated and transmit if lock, then wait until lock mark be set to end of transmission just perform described in from biography
The step of filename of the file data of end of transmission is obtained in defeated daily record.
According to existing transmission mechanism, the file data in source server is write to the process of purpose data sources unit
In, the data that the data and source server that are stored in purpose data sources unit itself are stored are compared by source server,
Judge the All Files data in source server whether are stored in purpose data sources unit, if it is not, then by purpose
The fractional transmission not stored in data sources unit gives purpose data sources unit.In the present invention, using by purpose data
The file data that source receiving unit is received is copied out, then the file data copied out is read out and handled.So, nothing
File data need to be read from purpose data sources unit to be handled, the number of files not received purpose data sources unit
According to being deleted, it is to avoid read/write conflict during file synchronization.
In practical application, Rsync can be used by realizing the software of file synchronization scheme of the present invention, and Rsync is a outstanding
File synchronization and delivery means under Unix system.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
God is with principle, and any modification, equivalent substitution and improvements done etc. should be included within the scope of protection of the invention.
Claims (8)
1. a kind of method of file synchronization, applied to destination server, it is characterised in that the destination server includes：Purpose
Data sources unit, purpose data source copy cell and data processing unit；This method includes：
The purpose data sources unit receives the file data of source server transmission, storage file data, and generates transmission
Daily record；Wherein, the file data is the file data that does not store in the purpose data sources unit；
The purpose data source copy cell obtains the filename of the file data received from transmission daily record, copies purpose number
According to the corresponding document data stored in the receiving unit of source, stored；
The data processing unit reads file data from purpose data source copy cell, carries out data processing, and delete described
The corresponding document data stored in purpose data source copy cell.
2. the method as described in claim 1, it is characterised in that during the purpose data sources unit generation transmission daily record,
This method also includes：Purpose data sources unit judges file data whether end of transmission, if it is, will lock mark set
For end of transmission, otherwise lock mark is set to transmit；
The purpose data source copy cell is obtained from transmission daily record before the filename of the file data received, this method
Also include：
Purpose data source copy cell is judged the lock mark, if lock is designated end of transmission, perform it is described from
The step of filename of the file data of end of transmission being obtained in transmission daily record；It is designated and transmits if lock, then
When lock mark is set to obtain the text of the file data of end of transmission during end of transmission just performs the daily record from transmission
The step of part name.
3. the method as described in claim 1, it is characterised in that the file data is user to each displaying content on webpage
Browse record.
4. a kind of system of file synchronization, it is characterised in that the system includes source server and destination server, the purpose clothes
Business device includes purpose data sources unit, purpose data source copy cell and data processing unit；
The source server, is not deposited for being transmitted to the purpose data sources unit in the purpose data sources unit
The file data of storage；
The purpose data sources unit, the file data for receiving the source server transmission, storage file data, and
Generation transmission daily record；
The purpose data source copy cell, the filename for obtaining the file data received from the transmission daily record,
The corresponding document data stored in copy purpose data sources unit, are stored；
The data processing unit, for reading file data from purpose data source copy cell, carries out data processing, and delete
The corresponding document data stored in the purpose data source copy cell.
5. system as claimed in claim 4, it is characterised in that the file data is user to each displaying content on webpage
Browse record.
6. a kind of equipment of file synchronization, it is characterised in that the equipment includes purpose data Src StUnit, purpose data source and copied
Shellfish unit and processing unit；
The purpose data sources unit, the file data for receiving source server transmission, storage file data, and generate
Transmit daily record；Wherein, the file data is the file data that does not store in the purpose data sources unit；
The purpose data source copy cell, the filename for obtaining the file data received from the transmission daily record,
The corresponding document data stored in copy purpose data sources unit, are stored；
The data processing unit, for reading file data from purpose data source copy cell, carries out data processing, and delete
The corresponding document data stored in purpose data source copy cell.
7. equipment as claimed in claim 6, it is characterised in that the purpose data sources unit includes lock mark and sets son
Unit, for generation transmission daily record when, judge file data whether end of transmission, if it is, will lock mark be set to biography
It is totally lost complete, otherwise is set to transmit by lock mark；
Purpose data source copy cell includes inquiry subelement and copy subelement；
The inquiry subelement, for judging the lock mark, if lock is designated end of transmission, to the copy
Subelement sends enabled instruction, is transmitted if lock is designated, and waits until that lock mark is set to end of transmission and just copied to described
Shellfish subelement sends enabled instruction；
The copy subelement, specifies for receiving described start, the number of files of end of transmission is obtained from transmission daily record
According to filename, the corresponding document data that store in copy purpose data sources unit are stored.
8. equipment as claimed in claims 6 or 7, it is characterised in that the file data is that user is interior to each displaying on webpage
That holds browses record.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110444163.5A CN103188282B (en) | 2011-12-27 | 2011-12-27 | Method, system and the equipment of file synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110444163.5A CN103188282B (en) | 2011-12-27 | 2011-12-27 | Method, system and the equipment of file synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103188282A CN103188282A (en) | 2013-07-03 |
| CN103188282B true CN103188282B (en) | 2017-08-04 |
Family
ID=48679247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110444163.5A Active CN103188282B (en) | 2011-12-27 | 2011-12-27 | Method, system and the equipment of file synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103188282B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794114B (en) * | 2014-01-16 | 2018-04-03 | 阿里巴巴集团控股有限公司 | A kind of data processing method and device |
| CN110309158B (en) * | 2018-04-13 | 2023-08-01 | 腾讯科技（深圳）有限公司 | Method and device for judging rolling abnormality of log file and readable medium |
| CN110059111B (en) * | 2019-04-17 | 2022-02-01 | 珠海金山网络游戏科技有限公司 | Data calling method and device, computing equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5742792A (en) * | 1993-04-23 | 1998-04-21 | Emc Corporation | Remote data mirroring |
| WO2001001249A1 (en) * | 1999-06-30 | 2001-01-04 | Computer Sciences Corporation | System and method for synchronizing copies of data in a computer system |
| EP1569119A2 (en) * | 1999-08-17 | 2005-08-31 | Emc Corporation | Remote mirroring system, device, and method |
| CN101640587A (en) * | 2008-08-01 | 2010-02-03 | 华为技术有限公司 | Data synchronization method and data synchronization device |
| CN101751307A (en) * | 2008-12-12 | 2010-06-23 | 比兹肯解决方法有限公司 | Data storage system, method as well as data storage and backup system |
| CN1998217B (en) * | 2004-05-28 | 2012-11-07 | 慕斯特有限公司 | System and method for replication, integration, consolidation and mobilization of data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050131969A1 (en) * | 2002-10-01 | 2005-06-16 | Fujitsu Limited | Database duplicating method, database duplicating apparatus, database creating method, and database creating apparatus |
-
2011
- 2011-12-27 CN CN201110444163.5A patent/CN103188282B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5742792A (en) * | 1993-04-23 | 1998-04-21 | Emc Corporation | Remote data mirroring |
| WO2001001249A1 (en) * | 1999-06-30 | 2001-01-04 | Computer Sciences Corporation | System and method for synchronizing copies of data in a computer system |
| EP1569119A2 (en) * | 1999-08-17 | 2005-08-31 | Emc Corporation | Remote mirroring system, device, and method |
| CN1998217B (en) * | 2004-05-28 | 2012-11-07 | 慕斯特有限公司 | System and method for replication, integration, consolidation and mobilization of data |
| CN101640587A (en) * | 2008-08-01 | 2010-02-03 | 华为技术有限公司 | Data synchronization method and data synchronization device |
| CN101751307A (en) * | 2008-12-12 | 2010-06-23 | 比兹肯解决方法有限公司 | Data storage system, method as well as data storage and backup system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103188282A (en) | 2013-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102891889B (en) | A kind of information sharing method and device | |
| EP3149899B1 (en) | Message attachment management | |
| US11102206B2 (en) | Methods and apparatus for establishing shared memory spaces for data access and distribution | |
| JP4859549B2 (en) | Information management method using management symbol and information management server | |
| CN104660635B (en) | Message synchronization method, device and system | |
| US20120226823A1 (en) | Document distribution system and method | |
| CN105357110B (en) | E-mail sending method, apparatus and system | |
| JP2015532473A (en) | Information processing method, apparatus, terminal, and server | |
| CN102789470A (en) | Method and device for uploading pictures in webpage | |
| JP2014149813A (en) | Method and system for sending e-mail with large file attached thereto on mobile device | |
| CN102857530B (en) | A kind of method and system of uploading pictures | |
| CN104079623B (en) | Multistage cloud storage synchronisation control means and system | |
| CN103188282B (en) | Method, system and the equipment of file synchronization | |
| JP5849639B2 (en) | Document management apparatus and document management system | |
| CN106375454A (en) | Method and system for uploading file by scanning graphic identifier | |
| CN110046000B (en) | Applet running method and device | |
| JP2015184989A (en) | Data supply device and data supply program | |
| CN105915613A (en) | Resource supply method and device based on cloud services | |
| CN104954398A (en) | Method and device for acquiring content | |
| CN105245446B (en) | A kind of file transmitting method and gateway | |
| CN105653566B (en) | A kind of method and device for realizing database write access | |
| US9633228B1 (en) | Verifiable media system and method | |
| CN113497787B (en) | File transmission method, equipment and system | |
| CN106230932A (en) | A kind of cloud disk shares the apparatus and method of file | |
| EP3151519A1 (en) | An intelligent system of unified content posting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |