WO2003030022A3 - Distributed file system and method of operating a distributed file system - Google Patents

Distributed file system and method of operating a distributed file system Download PDF

Info

Publication number
WO2003030022A3
WO2003030022A3 PCT/EP2002/011021 EP0211021W WO03030022A3 WO 2003030022 A3 WO2003030022 A3 WO 2003030022A3 EP 0211021 W EP0211021 W EP 0211021W WO 03030022 A3 WO03030022 A3 WO 03030022A3
Authority
WO
WIPO (PCT)
Prior art keywords
file system
file
distributed file
nodes
distributed
Prior art date
Application number
PCT/EP2002/011021
Other languages
French (fr)
Other versions
WO2003030022A2 (en
Inventor
Walter Tichy
Florin Isaila
Original Assignee
Partec Ag
Walter Tichy
Florin Isaila
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Partec Ag, Walter Tichy, Florin Isaila filed Critical Partec Ag
Priority to DE60206406T priority Critical patent/DE60206406T2/en
Priority to JP2003533157A priority patent/JP2005504396A/en
Priority to AT02800147T priority patent/ATE305635T1/en
Priority to EP02800147A priority patent/EP1440393B1/en
Priority to US10/491,459 priority patent/US7809774B2/en
Publication of WO2003030022A2 publication Critical patent/WO2003030022A2/en
Publication of WO2003030022A3 publication Critical patent/WO2003030022A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Multi Processors (AREA)
  • Remote Monitoring And Control Of Power-Distribution Networks (AREA)

Abstract

The present invention provides a distributed file system comprising a plurality of compute nodes and a plurality of input/output (I/O) nodes connected by an interconnection network wherein the system is adapted to use a common data representation for both physical and logical partitions of a file stored in the system and wherein the partitions are linearly addressable. The invention also provides a method of operating a distributed file system comprising a plurality of input/output (I/O) nodes and a plurality of compute nodes, the method comprising the steps of partitioning a file into a plurality of subfiles distributed across a plurality of I/O nodes; logically partioning a file by setting a view on it; computing mapping between a linear space of a file and a linear space of a subfile; computing the intersection between a view and and a subfile; and performing data operations.
PCT/EP2002/011021 2001-10-01 2002-10-01 Distributed file system and method of operating a distributed file system WO2003030022A2 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
DE60206406T DE60206406T2 (en) 2001-10-01 2002-10-01 DISTRIBUTED FILE SYSTEM AND METHOD FOR OPERATING A DISTRIBUTED FILE SYSTEM
JP2003533157A JP2005504396A (en) 2001-10-01 2002-10-01 Distributed file system and method of operating distributed file system
AT02800147T ATE305635T1 (en) 2001-10-01 2002-10-01 DISTRIBUTED FILE SYSTEM AND METHOD FOR OPERATING A DISTRIBUTED FILE SYSTEM
EP02800147A EP1440393B1 (en) 2001-10-01 2002-10-01 Distributed file system and method of operating a distributed file system
US10/491,459 US7809774B2 (en) 2001-10-01 2002-10-01 Distributed file system and method of operating a distributed file system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP01122698A EP1298536A1 (en) 2001-10-01 2001-10-01 Distributed file system and method of operating a distributed file system
EP01122698.2 2001-10-01

Publications (2)

Publication Number Publication Date
WO2003030022A2 WO2003030022A2 (en) 2003-04-10
WO2003030022A3 true WO2003030022A3 (en) 2003-10-16

Family

ID=8178688

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2002/011021 WO2003030022A2 (en) 2001-10-01 2002-10-01 Distributed file system and method of operating a distributed file system

Country Status (6)

Country Link
US (1) US7809774B2 (en)
EP (2) EP1298536A1 (en)
JP (2) JP2005504396A (en)
AT (1) ATE305635T1 (en)
DE (1) DE60206406T2 (en)
WO (1) WO2003030022A2 (en)

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050257827A1 (en) * 2000-04-27 2005-11-24 Russell Gaudiana Rotational photovoltaic cells, systems and methods
US20050015511A1 (en) * 2003-07-02 2005-01-20 Nec Laboratories America, Inc. Accelerated large data distribution in overlay networks
US7953819B2 (en) * 2003-08-22 2011-05-31 Emc Corporation Multi-protocol sharable virtual storage objects
DE102004019854B4 (en) * 2004-04-23 2010-09-16 Nec Europe Ltd. Data storage and access system
US7634566B2 (en) * 2004-06-03 2009-12-15 Cisco Technology, Inc. Arrangement in a network for passing control of distributed data between network nodes for optimized client access based on locality
US7389298B2 (en) * 2004-11-18 2008-06-17 International Business Machines Corporation Seamless remote traversal of multiple NFSv4 exported file systems
US8521752B2 (en) * 2005-06-03 2013-08-27 Osr Open Systems Resources, Inc. Systems and methods for arbitrary data transformations
US7870173B2 (en) * 2005-10-13 2011-01-11 International Business Machines Corporation Storing information in a common information store
US8713576B2 (en) * 2006-02-21 2014-04-29 Silicon Graphics International Corp. Load balancing for parallel tasks
JP4175379B2 (en) * 2006-04-25 2008-11-05 日本電気株式会社 File sharing method and file sharing system
EP2050002A2 (en) * 2006-08-01 2009-04-22 Massachusetts Institute of Technology Extreme virtual memory
US7512748B1 (en) 2006-08-17 2009-03-31 Osr Open Systems Resources, Inc. Managing lock rankings
US8539228B1 (en) 2006-08-24 2013-09-17 Osr Open Systems Resources, Inc. Managing access to a resource
CN101192227B (en) * 2006-11-30 2011-05-25 阿里巴巴集团控股有限公司 Log file analytical method and system based on distributed type computing network
US8024433B2 (en) * 2007-04-24 2011-09-20 Osr Open Systems Resources, Inc. Managing application resources
US7949693B1 (en) 2007-08-23 2011-05-24 Osr Open Systems Resources, Inc. Log-structured host data storage
US8326897B2 (en) * 2007-12-19 2012-12-04 International Business Machines Corporation Apparatus and method for managing data storage
US9565094B2 (en) * 2009-11-13 2017-02-07 International Business Machines Corporation I/O routing in a multidimensional torus network
US9954760B2 (en) 2010-01-29 2018-04-24 International Business Machines Corporation I/O routing in a multidimensional torus network
US9058334B2 (en) * 2010-02-11 2015-06-16 Emc Corporation Parallel file system processing
US8224879B2 (en) * 2010-02-23 2012-07-17 Hitachi, Ltd. Management system and management method for storage system
US8701113B2 (en) 2010-05-27 2014-04-15 International Business Machines Corporation Switch-aware parallel file system
US8903874B2 (en) 2011-11-03 2014-12-02 Osr Open Systems Resources, Inc. File system directory attribute correction
US9122700B2 (en) 2011-12-20 2015-09-01 Los Alamos National Security, Llc Parallel log structured file system collective buffering to achieve a compact representation of scientific and/or dimensional data
US9165014B1 (en) * 2012-06-28 2015-10-20 Emc Corporation Methods and apparatus for multi-resolution replication of files in a parallel computing system using semantic information
US9087075B1 (en) * 2012-06-28 2015-07-21 Emc Corporation Storing files in a parallel computing system using list-based index to identify replica files
US9229657B1 (en) 2012-11-01 2016-01-05 Quantcast Corporation Redistributing data in a distributed storage system based on attributes of the data
JP2014127015A (en) * 2012-12-26 2014-07-07 Toshiba Corp Information processor, distributed database system, and backup method
US9792295B1 (en) 2013-02-06 2017-10-17 Quantcast Corporation Distributing data of multiple logically independent file systems in distributed storage systems including physically partitioned disks
US9811529B1 (en) * 2013-02-06 2017-11-07 Quantcast Corporation Automatically redistributing data of multiple file systems in a distributed storage system
CN103235754B (en) * 2013-04-24 2016-10-05 曙光信息产业(北京)有限公司 The treating method and apparatus of request in distributed file system
US9767107B1 (en) * 2013-06-29 2017-09-19 EMC IP Holding Company LLC Parallel file system with metadata distributed across partitioned key-value store
US9811530B1 (en) * 2013-06-29 2017-11-07 EMC IP Holding Company LLC Cluster file system with metadata server for storage of parallel log structured file system metadata for a shared file
SG11201603105VA (en) * 2013-10-21 2016-05-30 Ab Initio Technology Llc Checkpointing a collection of data units
US9830329B2 (en) 2014-01-15 2017-11-28 W. Anthony Mason Methods and systems for data storage
WO2015116075A1 (en) 2014-01-30 2015-08-06 Hewlett-Packard Development Company, L.P. Copy message from application buffer to send buffer within kernel
US9483516B2 (en) * 2014-03-14 2016-11-01 Sap Se Multi-version concurrency control across row store and column store
US10162836B1 (en) * 2014-06-30 2018-12-25 EMC IP Holding Company LLC Parallel file system with striped metadata
CN107305490B (en) * 2016-04-22 2020-09-11 中国移动通信集团湖南有限公司 Metadata grouping method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5325526A (en) * 1992-05-12 1994-06-28 Intel Corporation Task scheduling in a multicomputer system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CORBETT P F ET AL: "OVERVIEW OF THE VESTA PARALLEL FILE SYSTEM", COMPUTER ARCHITECTURE NEWS, ASSOCIATION FOR COMPUTING MACHINERY, NEW YORK, US, vol. 21, no. 5, 1 December 1993 (1993-12-01), pages 7 - 14, XP000432532, ISSN: 0163-5964 *
CORBETT P F ET AL: "PARALLEL FILE SYSTEMS FOR THE IBM SP COMPUTERS", IBM SYSTEMS JOURNAL, IBM CORP. ARMONK, NEW YORK, US, vol. 34, no. 2, 21 March 1995 (1995-03-21), pages 222 - 246, XP000526617, ISSN: 0018-8670 *
LIGON W B ET AL: "IMPLEMENTATION AND PERFORMANCE OF A PARALLEL FILE SYSTEM FOR HIGH PERFORMANCE DISTRIBUTED APPLICATIONS", PROCEEDINGS OF THE INTERNATIONAL SYMPOSIUM OF HIGH PERFORMANCE DISTRIBUTED COMPUTING, XX, XX, August 1996 (1996-08-01), pages 471 - 480, XP002913597 *

Also Published As

Publication number Publication date
ATE305635T1 (en) 2005-10-15
JP4799669B2 (en) 2011-10-26
JP2010140507A (en) 2010-06-24
US20060101025A1 (en) 2006-05-11
WO2003030022A2 (en) 2003-04-10
DE60206406D1 (en) 2005-11-03
US7809774B2 (en) 2010-10-05
EP1440393B1 (en) 2005-09-28
EP1440393A2 (en) 2004-07-28
DE60206406T2 (en) 2006-06-14
EP1298536A1 (en) 2003-04-02
JP2005504396A (en) 2005-02-10

Similar Documents

Publication Publication Date Title
WO2003030022A3 (en) Distributed file system and method of operating a distributed file system
WO2004090788A3 (en) System and method for dynamically performing storage operations in a computer network
AU2001260916A1 (en) Control system for actuators in an aircraft
EP1507246A4 (en) Map data product, map data processing program product, map data processing method, and map data processing device
WO2001065371A3 (en) Method and system for updating an archive of a computer file
EP1734450A3 (en) Storage system and replication creation method thereof
CA2471841A1 (en) Integrated vehicle motion control system
WO2000027063A3 (en) Interface engine for managing business processes within a multimedia communication-center
AU1286197A (en) Overpartitioning system and method for increasing checkpoints in component-based parallel applications
WO2004044674A3 (en) Observation tool for signal processing components
EP1380944A3 (en) Method and system for automatic control of graphical computer application appearance and execution
GB2368430A (en) Method and apparatus for enhancing recovery operations in disk array storage devices
EP1085414A3 (en) Data migration method using storage area network (SAN)
WO2004021189A3 (en) Maintaining information using a plurality of storage attributes
WO2003003158A3 (en) Method and apparatus for instance based data transformation
WO2003025737A1 (en) Operation apparatus and operation system
WO2005026962A3 (en) Improvements in or relating to test systems or programs
WO2004072768A3 (en) Distributed dynamically optimizable processing communications and storage system
WO2004038588A3 (en) Hierarchical repository for configuration-related and performance-related information related to computerized systems
DE60036493D1 (en) DATA FLOW CONTROL
AU7087400A (en) System and method for sharing computer action scripts through a searchable database, and related applications thereof
WO1998043172A3 (en) Access control system
WO1998053399A3 (en) Operational analysis of software-controlled systems
AU2001285385A1 (en) Database systems, methods and computer program products including reconfigurable dynamic time window management
WO2003021516A1 (en) Performance predicting program, performance predicting device, and performance predicting method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BY BZ CA CH CN CO CR CU CZ DE DM DZ EC EE ES FI GB GD GE GH HR HU ID IL IN IS JP KE KG KP KR LC LK LR LS LT LU LV MA MD MG MN MW MX MZ NO NZ OM PH PL PT RU SD SE SG SI SK SL TJ TM TN TR TZ UA UG US UZ VC VN YU ZA ZM

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ UG ZM ZW AM AZ BY KG KZ RU TJ TM AT BE BG CH CY CZ DK EE ES FI FR GB GR IE IT LU MC PT SE SK TR BF BJ CF CG CI GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2003533157

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2002800147

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2002800147

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2006101025

Country of ref document: US

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 10491459

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2002800147

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 10491459

Country of ref document: US