CN103731369A - Method and system for updating data queue in server - Google Patents
Method and system for updating data queue in server Download PDFInfo
- Publication number
- CN103731369A CN103731369A CN201310741616.XA CN201310741616A CN103731369A CN 103731369 A CN103731369 A CN 103731369A CN 201310741616 A CN201310741616 A CN 201310741616A CN 103731369 A CN103731369 A CN 103731369A
- Authority
- CN
- China
- Prior art keywords
- data file
- described data
- internal queues
- local directory
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a method and system for updating a data queue in a server. According to the method and system, transmission errors generated when data files are written in a magnetic disk array can be avoided. The method comprises the steps that whether transaction happens to the data files in a local catalog or not is detected; if it is detected that the data files are newly added to the local catalog, an internal queue is established for the newly-added data files, and the internal queue is designated to record path information of the data files; if it is detected that the data files in the local catalog are modified, the path information in the queue is updated according to the path information of the data files; if it is detected that the data files in the local catalog are deleted, the internal queue corresponding to the data files is deleted.
Description
Technical field
The application relates to the method and system of transfer of data, is specifically related to update method and the system of the data queue in a kind of server.
Background technology
Along with the Fast Growth of network traffics, many services that are not easy in the past to realize also can be completed easily.For example: broadband upload multimedia file (multimedia file).Because the transmission of magnanimity still can affect the load of server.Therefore at known multimedia server, also can possess the function of Bandwidth Management (Bandwidth Manager).Server can see through the modes such as bandwidth distribution or buffer memory and alleviate the data transmission amount of same time.
In addition, described transmission means more combines multiple entity disks, and what make that data can be synchronous writes in described disk.And prior art writes the mode of disk to data, be that origin source synchronous data is write in server.Server can be kept in these data files.Afterwards, server by these data files in batches write entity disk.
Realizing of this mode writing is with low cost and quick.If but in the process that data write, make a mistake, data can only re-write.Therefore the time that data are transmitted again will be elongated, and can increase the load of the network bandwidth.In addition, same one data file is except will being again written into from client, and will again transmit in server inside.
And data file is to write in decentralized file device in the mode of buffer memory (buffer).Therefore all data files all can be discharged in same buffer memory.Confusion in the time of can causing like this data file to take out.Particularly, if can cause during different types of data file pointed mistake the meeting of two data files to damage.
Summary of the invention
The application's technical problem to be solved is to provide the update method of the data queue in a kind of server, the error of transmission of avoiding data file and occurring when cutting or while writing disk array.
In order to address the above problem, the update method that the application has disclosed the data queue in a kind of server comprises: whether the data file of detecting local directory has unusual fluctuation; If detect local directory while having newly-increased described data file, be that described newly-increased data file creates internal queues, and assign described internal queues to record the path information of described data file; If detect the described data file of local directory while being modified, according to the path information of described data file, upgrade the path information in internal queues; If detect the described data file of local directory when deleted, delete the internal queues of corresponding described data file.
Further, whether the data file of described detecting local directory has unusual fluctuation to comprise: record the establishment information of described internal queues and described data file, described establishment information comprises the time of reception of described data file, the file size of data file; Described in timing scan, whether data file is consistent with described establishment information; If when both are inconsistent, determine that the data file of local directory has unusual fluctuation.
Further, whether the data file of described detecting local directory has unusual fluctuation to comprise: if this described data file reading and there is not described data file relative recording time, define the unusual fluctuation of newly-increased described data file; If when this described data file reading and the time of reception of described data file last time reading and/or the file size of data file are inconsistent, define the unusual fluctuation of Update Table file; If last time the described data file of record and this while reading described in data file do not exist, define the unusual fluctuation of deleting data file.
Further, if detect the described data file of local directory while being modified, according to the path information of the described data file being modified, search internal queues corresponding to described data file, and upgrade the path information in the internal queues finding; If detect the described data file of local directory when deleted, according to the path information of deleted data file, search internal queues corresponding to deleted data file, and delete the internal queues finding.
In order to address the above problem, the application has separately disclosed a kind of transmission system of data cutting, comprising: whether detecting module, have unusual fluctuation for the data file of detecting local directory; Queue management module, be connected in described detecting module, if detect local directory while having newly-increased described data file for described detecting module, be described newly-increased data file establishment internal queues, and assign described internal queues to record the path information of described data file; Described queue management module, if when the described data file that also detects local directory for described detecting module is modified, upgrades the path information in internal queues according to the path information of described data file; If described detecting module detects the described data file of local directory when deleted, delete the internal queues of corresponding described data file.
Further, described detecting module further comprises: record cell, scanning element, determining unit; Wherein, described record cell, for recording the establishment information of described internal queues and described data file, described establishment information comprises the time of reception of described data file, the file size of data file; Whether described scanning element is consistent with described establishment information for data file described in timing scan; Described determining unit, if when inconsistent for both, determines that the data file of local directory has unusual fluctuation.
Further, described determining unit specifically for: if the described described data file that this reads and there is not described data file relative recording time, define the unusual fluctuation of newly-increased described data file; If when this described data file reading and the time of reception of described data file last time reading and/or the file size of data file are inconsistent, define the unusual fluctuation of Update Table file; If last time the described data file of record and this while reading described in data file do not exist, define the unusual fluctuation of deleting data file.
Further, described determining unit further comprises: internal queues is revised subelement, if while being modified for the described data file that detects local directory, according to the path information of the described data file being modified, search internal queues corresponding to described data file, and upgrade the path information in the internal queues finding; Internal queues is deleted subelement, if for detecting the described data file of local directory when deleted, search internal queues corresponding to deleted data file, and delete the internal queues finding according to the path information of deleted data file.
Compared with prior art, the application can obtain and comprise following technique effect:
1) the application improves the transmitting procedure of data file, if data file is write fashionable making a mistake, the application does not need again from client, to obtain data file.
2), if the application makes a mistake in during transmission data file, the application can see through at least one heartbeat server and transmit messages to Alarm Server, make that backstage personnel can be real-time to learn that what assembly occurs abnormal.
Certainly, arbitrary product of enforcement the application must not necessarily need to reach above-described all technique effects simultaneously.
Accompanying drawing explanation
Accompanying drawing described herein is used to provide further understanding of the present application, forms the application's a part, and the application's schematic description and description is used for explaining the application, does not form the improper restriction to the application.In the accompanying drawings:
Fig. 1 is the configuration diagram of the embodiment of the present application.
Fig. 2 is the operation workflow schematic diagram of the embodiment of the present application.
Embodiment
To coordinate drawings and Examples to describe the application's execution mode in detail below, by this application's implementation procedure how application technology means solve technical problem and reach technology effect can be fully understood and be implemented according to this.
As used some vocabulary in the middle of specification and claim, censure specific components.Those skilled in the art should understand, and hardware manufacturer may be called same assembly with different nouns.This specification and claims are not used as distinguishing the mode of assembly with the difference of title, but the difference in function is used as the criterion of distinguishing with assembly.If " comprising " mentioned in the middle of specification and claim is in the whole text an open language, therefore should be construed to " comprise but be not limited to "." roughly " refer to that in receivable error range, those skilled in the art can solve the technical problem within the scope of certain error, reach described technique effect substantially.In addition, " couple " word and comprise directly any and electric property coupling means indirectly at this.Therefore, if describe a first device in literary composition, be coupled to one second device, represent that described first device can directly be electrically coupled to described the second device, or be indirectly electrically coupled to described the second device by other devices or the means that couple.Specification subsequent descriptions is to implement the application's preferred embodiments, and right described description is to illustrate that the application's rule is object, not in order to limit the application's scope.The application's protection range is when being as the criterion depending on the claims person of defining.
Data queue in the application's server upgrades system, please refer to the 1st figure.The application's renewal system 100 is consisted of following element: memory modules 110, detecting module 120 and queue management module 130.Queue management module 130 is connected in detecting module 120 and memory modules 110.The data file that memory modules 110 is uploaded for receiving client.In memory modules 110, there are multiple local directories, each local directory storage data file separately.Whether detecting module 120 has unusual fluctuation for the data file of detecting local directory.If queue management module 130 detects local directory while having newly-increased described data file for described detecting module 120, be that described newly-increased data file creates internal queues, and assign described internal queues to record the path information of described data file.Queue management module 130, if when the described data file that also detects local directory for described detecting module 120 is modified, upgrades the path information in internal queues according to the path information of described data file.If described detecting module 120 detects the described data file of local directory when deleted, delete the internal queues of corresponding described data file.
The application's detecting module 120 further comprises that record cell 121, scanning element 122, determining unit 123, internal queues are revised subelement 1231 and internal queues is deleted subelement 1232.Record cell 121 is for recording the establishment information of described internal queues and described data file, and described establishment information comprises the time of reception of described data file, the file size of data file.Whether scanning element 122 is consistent with described establishment information for data file described in timing scan.If when determining unit 123 is inconsistent for both, determine that the data file of local directory has unusual fluctuation.
Particularly, if this described data file reading and there is not described data file relative recording time, determining unit 123 defines the unusual fluctuation of newly-increased described data file.If when this described data file reading and the time of reception of described data file last time reading and/or the file size of data file are inconsistent, determining unit 123 defines the unusual fluctuation of Update Table file.If last time the described data file of record and this while reading described in data file do not exist, determining unit 123 defines the unusual fluctuation of deleting data file.
If when internal queues modification subelement 1231 is modified for detecting the described data file of local directory, according to the path information of the described data file being modified, search internal queues corresponding to described data file, and upgrade the path information in the internal queues finding.Internal queues is deleted subelement 1232, if for detecting the described data file of local directory when deleted, search internal queues corresponding to deleted data file, and delete the internal queues finding according to the path information of deleted data file.
At this, function mode of aforementioned each assembly is represented with following step, please coordinate shown in the 2nd figure:
Step S210: whether the data file of detecting local directory has unusual fluctuation;
Step S220: if detect local directory while having newly-increased described data file, be that described newly-increased data file creates internal queues, and assign described internal queues to record the path information of described data file;
Step S230: if detect the described data file of local directory while being modified, upgrade the path information in internal queues according to the path information of described data file; And
Step S240: if detect the described data file of local directory when deleted, delete the internal queues of corresponding described data file.
Whether the application has in the step of unusual fluctuation further comprising the steps for the data file of detecting local directory:
Step S310: record the establishment information of described internal queues and described data file, described establishment information comprises the time of reception of described data file, the file size of data file;
Step S320: described in timing scan, whether data file is consistent with described establishment information;
Step S330: if when both are inconsistent, determine that the data file of local directory has unusual fluctuation.
In step S310, file operation function can regularly be called read the data file of local directory, and log file name, the time of reception of data file, the file size of data file etc. create information.Also can regularly monitor the read-write operation of file operation function to local directory, thereby obtain file unusual action information.
In step S320 and in S330, the document creation information of timing scan record, by once document creation information and this document creation information reading of record before relatively, has judged whether file unusual fluctuation.If relatively the document creation information of twi-read (as the file size of the time of reception of filename, data, data any one) inconsistent, think and have data file unusual fluctuation.
In above-mentioned steps S320 and step S330: if this described data file reading and there is not described data file relative recording time, define the unusual fluctuation of newly-increased described data file;
If (as change has occurred for the time of reception of data file and/or file size), defines the unusual fluctuation of Update Table file when this described data file reading and the time of reception of described data file last time reading and/or the file size of data file are inconsistent;
If last time the described data file of record and this while reading described in data file do not exist, the unusual fluctuation that defines deletion data file (had been recorded the name of this data file while last time reading, in the record reading at this, through relatively, do not find the name of this data file).
When there is the reformed unusual fluctuation of data file, according to the path information of the described data file being modified, search internal queues corresponding to described data file, and upgrade the path information in the internal queues finding; If when there is to detect the deleted unusual fluctuation of the described data file of local directory, search internal queues corresponding to deleted data file according to the path information of deleted data file, and delete the internal queues finding.During concrete enforcement, can internal queues corresponding to specified data file by the routing information stored in internal queues relatively; Also can, when creating internal queues, create a queue index file, for the corresponding relation of save data file and internal queues simultaneously.The routing information of data file can comprise that described data file is stored in the path of local directory, conventionally by the interface function of system, can obtain; Also can comprise and be stored in the path of end in server, the store path information of include file in common name, as the concrete memory disk position on the IP address of server and server etc.
Also it should be noted that, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby the process, method, commodity or the system that make to comprise a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or be also included as the intrinsic key element of this process, method, commodity or system.The in the situation that of more restrictions not, the key element being limited by statement " comprising ... ", and be not precluded within process, method, commodity or the system that comprises described key element and also have other identical element.
The embodiment that it will be understood by those skilled in the art that the application can be provided as method, system or computer program.Therefore, the application can adopt complete hardware implementation example, completely implement software example or the form in conjunction with the embodiment of software and hardware aspect.And the application can adopt the form at one or more upper computer programs of implementing of computer-usable storage medium (including but not limited to magnetic disc store, CD-ROM, optical memory etc.) that wherein include computer usable program code.
The embodiment that the foregoing is only the application, is not limited to the application.To those skilled in the art, the application can have various modifications and variations.Any modification of doing within all spirit in the application and principle, be equal to replacement, improvement etc., within all should being included in the application's claim scope.
Claims (8)
1. a update method for the data queue in server, is characterized in that, comprises the following steps:
Whether the data file of detecting local directory has unusual fluctuation;
If detect local directory while having newly-increased described data file, be that described newly-increased data file creates internal queues, and assign described internal queues to record the path information of described data file;
If detect the described data file of local directory while being modified, according to the path information of described data file, upgrade the path information in internal queues;
And, if detect the described data file of local directory when deleted, delete the internal queues of corresponding described data file.
2. the update method of the data queue in server as claimed in claim 1, is characterized in that, whether the data file of described detecting local directory has unusual fluctuation, further comprises:
Record the establishment information of described internal queues and described data file, described establishment information comprises the time of reception of described data file, the file size of data file;
Described in timing scan, whether data file is consistent with described establishment information; And
If when both are inconsistent, determine that the data file of local directory has unusual fluctuation.
3. the update method of the data queue in server as claimed in claim 2, is characterized in that, whether the data file of described detecting local directory has unusual fluctuation, further comprises:
If when the described data file that this reads and the relative recording that does not have described data file, define the unusual fluctuation of newly-increased described data file;
If when this described data file reading and the time of reception of described data file last time reading and/or the file size of data file are inconsistent, define the unusual fluctuation of Update Table file;
If last time the described data file of record and this while reading described in data file do not exist, define the unusual fluctuation of deleting data file.
4. the update method of the data queue in server as claimed in claim 1, it is characterized in that, if detect the described data file of local directory while being modified, according to the path information of the described data file being modified, search internal queues corresponding to described data file, and upgrade the path information in the internal queues finding;
And, if detect the described data file of local directory when deleted, according to the path information of deleted data file, search internal queues corresponding to deleted data file, and delete the internal queues finding.
5. a updating device for the data queue in server, is characterized in that comprising:
Whether detecting module, have unusual fluctuation for the data file of detecting local directory; And,
Queue management module, be connected in described detecting module, if detect local directory while having newly-increased described data file for described detecting module, be described newly-increased data file establishment internal queues, and assign described internal queues to record the path information of described data file;
Described queue management module, if when the described data file that also detects local directory for described detecting module is modified, upgrades the path information in internal queues according to the path information of described data file;
And, if described detecting module detects the described data file of local directory when deleted, delete the internal queues of corresponding described data file.
6. the renewal system of the data queue in server as claimed in claim 5, is characterized in that, described detecting module further comprises: record cell, scanning element, determining unit; Wherein,
Described record cell, for recording the establishment information of described internal queues and described data file, described establishment information comprises the time of reception of described data file, the file size of data file;
Whether described scanning element is consistent with described establishment information for data file described in timing scan; And,
Described determining unit, if when inconsistent for both, determines that the data file of local directory has unusual fluctuation.
7. the renewal system of the data queue in server as claimed in claim 6, is characterized in that, described determining unit specifically for:
If the described described data file that this reads and there is not described data file relative recording time, define the unusual fluctuation of newly-increased described data file;
If when this described data file reading and the time of reception of described data file last time reading and/or the file size of data file are inconsistent, define the unusual fluctuation of Update Table file;
If last time the described data file of record and this while reading described in data file do not exist, define the unusual fluctuation of deleting data file.
8. the renewal system of the data queue in server as claimed in claim 5, is characterized in that, described determining unit further comprises:
Internal queues is revised subelement, if while being modified for the described data file that detects local directory, according to the path information of the described data file being modified, search internal queues corresponding to described data file, and upgrade the path information in the internal queues finding;
And internal queues is deleted subelement, if for detecting the described data file of local directory when deleted, search internal queues corresponding to deleted data file according to the path information of deleted data file, and delete the internal queues finding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310741616.XA CN103731369A (en) | 2013-12-27 | 2013-12-27 | Method and system for updating data queue in server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310741616.XA CN103731369A (en) | 2013-12-27 | 2013-12-27 | Method and system for updating data queue in server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103731369A true CN103731369A (en) | 2014-04-16 |
Family
ID=50455301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310741616.XA Pending CN103731369A (en) | 2013-12-27 | 2013-12-27 | Method and system for updating data queue in server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103731369A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1091508A (en) * | 1996-09-12 | 1998-04-10 | Toshiba Corp | Preserved data updating device |
| US20070061540A1 (en) * | 2005-06-06 | 2007-03-15 | Jim Rafert | Data storage system using segmentable virtual volumes |
| CN101237595A (en) * | 2008-02-25 | 2008-08-06 | 中兴通讯股份有限公司 | Data load method |
| CN101902473A (en) * | 2010-07-22 | 2010-12-01 | 武汉中地数码科技有限公司 | Method for synchronously updating data based on grid GIS (Geographic Information System) |
| CN102004778A (en) * | 2010-11-19 | 2011-04-06 | 清华大学 | Text index online updating method in cloud environment |
| US8150837B2 (en) * | 2004-06-25 | 2012-04-03 | Apple Inc. | Methods and systems for managing data |
| CN102467412A (en) * | 2010-11-16 | 2012-05-23 | 金蝶软件（中国）有限公司 | Method, device and business system for processing operation request |
| US20120167111A1 (en) * | 2010-12-23 | 2012-06-28 | Microsoft Corporation | Resource deployment based on conditions |
| CN102779185A (en) * | 2012-06-29 | 2012-11-14 | 浙江大学 | High-availability distribution type full-text index method |
| CN103020315A (en) * | 2013-01-10 | 2013-04-03 | 中国人民解放军国防科学技术大学 | Method for storing mass of small files on basis of master-slave distributed file system |
-
2013
- 2013-12-27 CN CN201310741616.XA patent/CN103731369A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1091508A (en) * | 1996-09-12 | 1998-04-10 | Toshiba Corp | Preserved data updating device |
| US8150837B2 (en) * | 2004-06-25 | 2012-04-03 | Apple Inc. | Methods and systems for managing data |
| US20070061540A1 (en) * | 2005-06-06 | 2007-03-15 | Jim Rafert | Data storage system using segmentable virtual volumes |
| CN101237595A (en) * | 2008-02-25 | 2008-08-06 | 中兴通讯股份有限公司 | Data load method |
| CN101902473A (en) * | 2010-07-22 | 2010-12-01 | 武汉中地数码科技有限公司 | Method for synchronously updating data based on grid GIS (Geographic Information System) |
| CN102467412A (en) * | 2010-11-16 | 2012-05-23 | 金蝶软件（中国）有限公司 | Method, device and business system for processing operation request |
| CN102004778A (en) * | 2010-11-19 | 2011-04-06 | 清华大学 | Text index online updating method in cloud environment |
| US20120167111A1 (en) * | 2010-12-23 | 2012-06-28 | Microsoft Corporation | Resource deployment based on conditions |
| CN102779185A (en) * | 2012-06-29 | 2012-11-14 | 浙江大学 | High-availability distribution type full-text index method |
| CN103020315A (en) * | 2013-01-10 | 2013-04-03 | 中国人民解放军国防科学技术大学 | Method for storing mass of small files on basis of master-slave distributed file system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10430398B2 (en) | Data storage system having mutable objects incorporating time | |
| US20210173853A1 (en) | Selective synchronization of content items in a content management system | |
| US7730071B2 (en) | Data management system and data management method | |
| CN100588236C (en) | Data reproducing device and content management method | |
| US11687488B2 (en) | Directory deletion method and apparatus, and storage server | |
| US9575974B2 (en) | Distributed file system gateway | |
| US20070299880A1 (en) | Document Management Server, Document Management Method, Computer Readable Medium, Computer Data Signal, and System For Managing Document Use | |
| US10013312B2 (en) | Method and system for a safe archiving of data | |
| US20160259694A1 (en) | Method and device for organizing and restoring file indexeses | |
| CN103793479A (en) | Log management method and log management system | |
| KR20160004285A (en) | File management with placeholders | |
| CN102272751B (en) | Data integrity in a database environment through background synchronization | |
| US20190121793A1 (en) | Data updating method and device for a distributed database system | |
| CN105612518A (en) | Methods and systems for autonomous memory searching | |
| US20140136496A1 (en) | System, method and non-transitory computer readable storage medium for supporting network file accessing and versioning with multiple protocols in a cloud storage server | |
| US20110040788A1 (en) | Coherent File State System Distributed Among Workspace Clients | |
| US8370532B1 (en) | Techniques for portable device data archiving | |
| CN105765570A (en) | Music identification | |
| CN103605479B (en) | Data file wiring method and system, data file read method and system | |
| US20120005162A1 (en) | Managing Copies of Data Structures in File Systems | |
| CN103731369A (en) | Method and system for updating data queue in server | |
| CN103701667A (en) | Method, device and system for monitoring heartbeat of server | |
| US8615491B2 (en) | Archiving tool for managing electronic data | |
| US20130218851A1 (en) | Storage system, data management device, method and program | |
| US20170262439A1 (en) | Information processing apparatus and non-transitory computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20151225 Address after: Room six, building 19, building 68, No. 100089 South Road, Haidian District, Beijing Applicant after: LETV CLOUD COMPUTING CO., LTD. Address before: Room six, building 19, building 68, No. 100089 South Road, Haidian District, Beijing Applicant before: LeTV Information Technology (Beijing) Co., Ltd. |
|
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20180112 |
|
| AD01 | Patent right deemed abandoned |