WO2002061737A3 - Dynamically distributed file system - Google Patents

Dynamically distributed file system Download PDF

Info

Publication number
WO2002061737A3
WO2002061737A3 PCT/US2002/002702 US0202702W WO02061737A3 WO 2002061737 A3 WO2002061737 A3 WO 2002061737A3 US 0202702 W US0202702 W US 0202702W WO 02061737 A3 WO02061737 A3 WO 02061737A3
Authority
WO
WIPO (PCT)
Prior art keywords
file system
server
file
file server
system information
Prior art date
Application number
PCT/US2002/002702
Other languages
French (fr)
Other versions
WO2002061737A9 (en
WO2002061737A8 (en
WO2002061737A2 (en
Inventor
John R Staub
Original Assignee
Snap Appliance Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Snap Appliance Inc filed Critical Snap Appliance Inc
Priority to AU2002242026A priority Critical patent/AU2002242026A1/en
Publication of WO2002061737A2 publication Critical patent/WO2002061737A2/en
Publication of WO2002061737A9 publication Critical patent/WO2002061737A9/en
Publication of WO2002061737A3 publication Critical patent/WO2002061737A3/en
Publication of WO2002061737A8 publication Critical patent/WO2002061737A8/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
    • G06F11/2035—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant without idle spare hardware
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2094—Redundant storage or storage space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2097—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements maintaining the standby controller/processing unit updated
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00—Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10—Indexing scheme relating to G06F11/10
    • G06F2211/1002—Indexing scheme relating to G06F11/1076
    • G06F2211/1028—Distributed, i.e. distributed RAID systems with parity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A dynamically distributed file system is described. The dynamically distributed file system operates on a computer network (110) and includes a first file server (130) that is operably connected to a network fabric (120) and a second file server (131) that is operably connected to the network fabric (120). The dynamically distributed file system further includes first file system information loaded on said first file server and second file system information loaded on said second file server. The first file system information and the second file system information are configured to allow a client computer that is operably connected to the network fabric to locate files stored by the first file server and files stored by the second file server without prior knowledge as to which file server stores the files. The use of distributed parity groups allows data to be stored in a RAID-type disk array (140-141) without the limitations of conventional RAID. In one embodiment, performance monitoring is used to move files from one server to another to improve system efficiency. In one embodiment, a data engine offloads data movement workload from a server CPU.
PCT/US2002/002702 2001-01-29 2002-01-29 Dynamically distributed file system WO2002061737A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2002242026A AU2002242026A1 (en) 2001-01-29 2002-01-29 Dynamically distributed file system

Applications Claiming Priority (14)

Application Number Priority Date Filing Date Title
US26469401P 2001-01-29 2001-01-29
US26467201P 2001-01-29 2001-01-29
US26466801P 2001-01-29 2001-01-29
US26467001P 2001-01-29 2001-01-29
US26467101P 2001-01-29 2001-01-29
US26466901P 2001-01-29 2001-01-29
US26467301P 2001-01-29 2001-01-29
US60/264,670 2001-01-29
US60/264,668 2001-01-29
US60/264,673 2001-01-29
US60/264,671 2001-01-29
US60/264,694 2001-01-29
US60/264,669 2001-01-29
US60/264,672 2001-01-29

Publications (4)

Publication Number Publication Date
WO2002061737A2 WO2002061737A2 (en) 2002-08-08
WO2002061737A9 WO2002061737A9 (en) 2002-11-21
WO2002061737A3 true WO2002061737A3 (en) 2003-07-31
WO2002061737A8 WO2002061737A8 (en) 2004-02-12

Family

ID=27569513

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2002/002702 WO2002061737A2 (en) 2001-01-29 2002-01-29 Dynamically distributed file system

Country Status (2)

Country Link
AU (1) AU2002242026A1 (en)
WO (1) WO2002061737A2 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7146524B2 (en) 2001-08-03 2006-12-05 Isilon Systems, Inc. Systems and methods for providing a distributed file system incorporating a virtual hot spare
US7937421B2 (en) 2002-11-14 2011-05-03 Emc Corporation Systems and methods for restriping files in a distributed file system
US7386675B2 (en) 2005-10-21 2008-06-10 Isilon Systems, Inc. Systems and methods for using excitement values to predict future access to resources
US7346720B2 (en) 2005-10-21 2008-03-18 Isilon Systems, Inc. Systems and methods for managing concurrent access requests to a shared resource
JP2007206931A (en) * 2006-02-01 2007-08-16 Hitachi Ltd Storage system, data processing method and storage device
JP5057741B2 (en) 2006-10-12 2012-10-24 株式会社日立製作所 Storage device
US8966080B2 (en) 2007-04-13 2015-02-24 Emc Corporation Systems and methods of managing resource utilization on a threaded computer system
KR100977159B1 (en) * 2007-12-14 2010-08-20 한국전자통신연구원 Method And System Of File Metadata Management Transparent About Address Changing Of Data Servers And Their Disk Movements
CN107533518B (en) 2015-01-20 2020-09-18 乌尔特拉塔有限责任公司 Distributed index for fault tolerant object storage fabric
US11579774B2 (en) 2015-01-20 2023-02-14 Ultrata, Llc Object memory data flow triggers
US9886210B2 (en) 2015-06-09 2018-02-06 Ultrata, Llc Infinite memory fabric hardware implementation with router
WO2017100281A1 (en) 2015-12-08 2017-06-15 Ultrata, Llc Memory fabric software implementation
US10671460B2 (en) 2018-02-05 2020-06-02 Micron Technology, Inc. Memory access communications through message passing interface implemented in memory systems
CN111492643B (en) * 2018-03-02 2021-10-15 华为技术有限公司 Apparatus and method for code caching
US10942808B2 (en) 2018-12-17 2021-03-09 International Business Machines Corporation Adaptive data and parity placement using compression ratios of storage devices
WO2023227945A1 (en) * 2022-05-25 2023-11-30 Neuroblade Ltd. Processing systems and methods

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4837680A (en) * 1987-08-28 1989-06-06 International Business Machines Corporation Controlling asynchronously operating peripherals
US5566331A (en) * 1994-01-24 1996-10-15 University Corporation For Atmospheric Research Mass storage system for file-systems
US5907672A (en) * 1995-10-04 1999-05-25 Stac, Inc. System for backing up computer disk volumes with error remapping of flawed memory addresses
US5931947A (en) * 1997-09-11 1999-08-03 International Business Machines Corporation Secure array of remotely encrypted storage devices
US5974503A (en) * 1997-04-25 1999-10-26 Emc Corporation Storage and access of continuous media files indexed as lists of raid stripe sets associated with file names

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4837680A (en) * 1987-08-28 1989-06-06 International Business Machines Corporation Controlling asynchronously operating peripherals
US5566331A (en) * 1994-01-24 1996-10-15 University Corporation For Atmospheric Research Mass storage system for file-systems
US5907672A (en) * 1995-10-04 1999-05-25 Stac, Inc. System for backing up computer disk volumes with error remapping of flawed memory addresses
US5974503A (en) * 1997-04-25 1999-10-26 Emc Corporation Storage and access of continuous media files indexed as lists of raid stripe sets associated with file names
US5931947A (en) * 1997-09-11 1999-08-03 International Business Machines Corporation Secure array of remotely encrypted storage devices

Also Published As

Publication number Publication date
WO2002061737A9 (en) 2002-11-21
WO2002061737A8 (en) 2004-02-12
WO2002061737A2 (en) 2002-08-08
AU2002242026A1 (en) 2002-08-12

Similar Documents

Publication Publication Date Title
WO2002061737A3 (en) Dynamically distributed file system
Al-Kiswany et al. Vmflock: Virtual machine co-migration for the cloud
US8195628B2 (en) Method and system for data reduction
Anderson et al. Interposed request routing for scalable network storage
US7099900B1 (en) Mapping driver for use in data backup systems
JP5068252B2 (en) Data placement technology for striping data containers across multiple volumes in a storage system cluster
US7720889B1 (en) System and method for nearly in-band search indexing
Zhu et al. HBA: Distributed metadata management for large cluster-based storage systems
WO2001082058A3 (en) Dynamic file system configurations
Vazhkudai et al. Freeloader: Scavenging desktop storage resources for scientific data
WO2005119494A3 (en) Transaction based virtual file system optimized for high-latency network connections
WO2004061605A3 (en) Medata based file switch and switched file system
EP1126374A3 (en) Masking logical unit numbers in a shared data storage system
CA2171572A1 (en) System and Method for Determining and Manipulating Configuration Information of Servers in a Distributed Object Environment
TW200401973A (en) Virtual one-dimensional method and device of multiple network storages
CA2604742C (en) Method for distributing computing between server and client
US20070266145A1 (en) Method, system and program product for collecting web metric data
WO2001025908A3 (en) Presentation service architectures for netcentric computing systems
CA2420422A1 (en) System and method for data management
US20200050472A1 (en) Methods and apparatus for providing hypervisor level data services for server virtualization
MY147118A (en) High performance computing system and method
EP1369785A3 (en) Management server for a plurality of disk arrays
EP1536336A3 (en) Method, system and server for performing backup in a computer system with multiple file servers
US7200716B1 (en) Method and apparatus to offload operations in a networked storage system
Welch et al. Managing Scalability in Object Storage Systems for HPC Linux Clusters.

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SL TJ TM TN TR TT TZ UA UG UZ VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: C2

Designated state(s): AE AG AL AM AT AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ CZ DE DE DK DK DM DZ EC EE EE ES FI FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SK SL TJ TM TN TR TT TZ UA UG UZ VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: C2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

COP Corrected version of pamphlet

Free format text: PAGES 1/46-46/46, DRAWINGS, REPLACED BY NEW PAGES 1/46-46/46; DUE TO LATE TRANSMITTAL BY THE RECEIVING OFFICE

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: COMMUNICATION UNDER RULE 69 EPC (FORM 1205A OF 05.11.2003)

CFP Corrected version of a pamphlet front page
CR1 Correction of entry in section i

Free format text: IN PCT GAZETTE 32/2002 DUE TO A TECHNICAL PROBLEMAT THE TIME OF INTERNATIONAL PUBLICATION, SOME INFORMATION WAS MISSING UNDER (81). THE MISSING INFORMATION NOW APPEARS IN THE CORRECTED VERSION

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP