Difference between revisions of "Storage"

From Vogons Wiki
Jump to: navigation, search
(→‎Know your target)
(→‎Overcoming Limitations and Incompatibilities)
 
(141 intermediate revisions by the same user not shown)
Line 1: Line 1:
  
−
===What controller will you use?===
+
===Common IDE Categories===
−
'''ISA IDE'''
+
'''Early IDE (ATA-0 & ATA-1) '''
−
* 16bit ISA controllers, common before 1995, that essentially connected IDE storage directly to the ISA bus
+
* Commonly used with 286, 386, and early 486 computers
−
* Interface is slow enough that all storage built since 1995 connected to it will have similar performance
+
* IDE for 16bit ISA controllers, connects IDE storage devices directly to the ISA/AT bus
−
* Modern storage is very compatible with these adapters
+
* Storage transfer rates depend on the clock speed of the ISA bus
−
* 528MB limit - BIOS before July 1994 rarely support drives over 528MB (IDE Heads <= 16, BIOS Sectors <= 63, BIOS Cylinders <= 1024 )
+
* Most devices use PIO transfers.
−
'''Early EIDE'''
+
* DMA support is extremely rare because it was not faster than PIO on an ISA bus.
−
* Common between 1993-1996
+
−
* Mid to Late 486
+
−
* VLB without BIOS support
+
−
* 528MB limit - BIOS before July 1994 rarely support drives over 528MB (IDE Heads <= 16, BIOS Sectors <= 63, BIOS Cylinders <= 1024 )
+
−
* 2GB limit - BIOS before May 1996 rarely support drives larger than 2015 MB ( BIOS cylinder values <= 4095 )
+
−
* Phoenix BIOS 4.03 and 4.04 lock when a drive is configured with a capacity over 3277 MB.
+
−
'''Changing ATA (ATA-33 & ATA-66)'''
+
−
* Common between 1995-1998
+
−
* Late 486 and Pentium Socket 7 systems
+
−
* 2GB limit - BIOS before May 1996 rarely support drives larger than 2015 MB ( BIOS cylinder values <= 4095 )
+
−
* 8GB limit - DOS and Windows 95/98 are limited to 255 Heads.
+
−
'''Mature ATA (ATA-100 & ATA-133)'''
+
−
* Common between 1999-2010
+
−
* Little performance difference between the two protocols
+
−
* Require connection to the CPU faster PCI to reach full potential
+
−
* BIOS before 2002 rarely supports 128GB drives
+
−
'''Sata IDE'''
+
  
 +
'''XT-IDE '''
 +
* This is a card that allows 16 bit IDE devices to work in an 8 bit slot
 +
* Also used to describe storage devices that support a rarely used 8 bit transfer protocol
 +
* Also used as shorthand for XtIDE Universal BIOS, which is an ISA Option ROM for PC's and not drive controller or storage device.
  
−
===IDE and BIOS Limits===
+
'''EIDE, Fast ATA (ATA-2 & ATA-3 )'''
−
* 528 MB limit - BIOS before July 1994 rarely support drives over 528MB (ATA0 Heads <= 16, BIOS Sectors <= 63, BIOS Cylinders <= 1024 )
+
* Common between 1994-1997: Late Socket 3, Socket 4,5,6, early Socket 7
−
* 2015 MB limit - BIOS before May 1996 rarely support drives larger than 2015 MB ( BIOS cylinder values <= 4095 )
+
* PIO modes transfer rates up to 16Mhz available on some VLB controllers and most PCI controllers
−
* 3277 MB limit - Phoenix BIOS 4.03 and 4.04 lock when a drive is configured with a capacity over 3277 MB.  
+
* Multiword DMA transfer rates to 16Mhz available on a few VLB controllers and most PCI controllers.
−
* 4.2 GB limit - BIOS before Feb 1997 have the ECHS (Extended CHS) limit.  DOS and Windows 95/98 cannot handle 256 heads.  'Large' mode in the BIOS produces an alternate geometry by doubling the number of heads and halving the number of cylinders shown to DOS to keep the cylinders below 1024. This method stops working at 4032 MB (1024 cylinders, 128 heads, 63 sectors) if the drive reports 16 heads.
+
* Controllers from this period were often paired with BIOS support for storage >512MB using ECHS addressing
−
* 7.9 GB limit - Revised ECHS limit.  'Large' mode in the BIOS by presenting an alternate geometry using multiples of 15 heads so that 240 heads can be presented as the max alternate geometry. This method stops working at 7560 MB (1024 cylinders, 240 heads, 63 sectors)
+
* VLB controllers from this period sometimes have jumpers that allow transfer speed to be configured without a driver.
−
*
+
* VLB controllers from this period sometimes have a jumper to allow the use of IO Ready signaling to increase the efficiency of PIO modes, but this requires driver support.
−
8.4 GB limit
+
−
-
+
−
w4 GB limit
+
−
Finally, if the BIOS does all it can to make this translation a success, and uses 255 heads and 63 sectors/track (`assisted LBA' or just `LBA') it may reach 1024*255*63*512=8422686720 bytes, slightly less than the earlier 8.5 GB limit because the geometries with 256 heads must be avoided. (This translation will use for the number of heads the first value H in the sequence 16, 32, 64, 128, 255 for which the total disk capacity fits in 1024*H*63*512, and then computes the number of cylinders C as total capacity divided by (H*63*512).)
+
  
−
The 33.8 GB limit (August 1999)
+
'''ATA-4, UDMA/33, Ultra ATA/33'''
−
The next hurdle comes with a size over 33.8 GB. The problem is that with the default 16 heads and 63 sectors/track this corresponds to a number of cylinders of more than 65535, which does not fit into a short. Many BIOSes couldn't handle such disks. (See, e.g., Asus upgrades for new flash images that work.) Linux kernels older than 2.2.14 / 2.3.21 need a patch. See IDE problems with 34+ GB disks below.
+
* Common between 1998-1999: Pentium II & III, Super Socket 7, Early Athlon
 +
* UDMA2 operated at 33MHz for significant performance increases over previous generations
 +
* 80 connector IDE cables recommended but not required
 +
* UDMA bus master transfer modes significantly improve performance in multitasking operating systems
 +
* Few BIOS directly allow UDMA transfers in DOS without a UMDA driver in config.sys
 +
* Windows 98 requires users to change a setting in Device manager to enable UDMA Bus mastering
 +
* Some Award BIOS from this period require a patch to allow UDMA operation in Win98. https://www.vogons.org/viewtopic.php?p=1272522#p1272522
 +
* Controllers from this period were often paired with BIOS support that support >8.4GB using LBA28 addressing
  
−
The 137 GB limit (Sept 2001)
+
'''Mature PATA (ATA-5, ATA-6, ATA-7: Ultra ATA-66, ATA-100 & ATA-133)'''
−
As mentioned above, the old ATA protocol uses 16+4+8 = 28 bits to specify the sector number, and hence cannot address more than 2^28 sectors. ATA-6 describes an extension that allows the addressing of 2^48 sectors, a million times as much. There is support in very recent kernels.
+
* Common between 2000-2010: Later Pentium 3 & Athlon through Core2
 +
* UDMA4 & UDMA5 & UDMA6 require an 80 connector IDE cable.  An 80 connector can be simulated by grounding Pin #34 on the storage device.  https://www.vogons.org/viewtopic.php?t=77881
 +
* UDMA5 & UDMA6 rarely show significant performance improvement over UDMA4 unless paired with the fastest PATA storage devices and a south bridge that uses >= Intel ICH5, VIA Enhanced-Vlink or AMD HyperTransport.  
 +
* Controllers from this period were often paired with BIOS support for storage >128GB using LBA48
  
−
The 2 TiB limit
+
More information about PATA ATA is available here: https://en.wikipedia.org/wiki/Parallel_ATA
−
With 32-bit sector numbers, one can address 2 TiB. A lot of software will have to be rewritten once disks get larger.
+
  
−
Hard drives over 8.4 GB are supposed to report their geometry as 16383/16/63. This in effect means that the `geometry' is obsolete, and the total disk size can no longer be computed from the geometry, but is found in the LBA capacity field returned by the IDENTIFY command. Hard drives over 137.4 GB are supposed to report an LBA capacity of 0xfffffff = 268435455 sectors (137438952960 bytes). Now the actual disk size is found in the new 48-capacity field.
+
'''SATA'''
 +
* Common after 2004
 +
* Early SATA devices were often PATA devices with a built in SATA bridge
 +
* Bridges are commonly available today to connect PATA devices to SATA controllers & vice versa.
 +
* Most native SATA devices support ACHI features such as NCQ, TRIM, better power management and hot swapping, but these features can be difficult or impossible to access when the device is attached through a PATA bridge.
 +
* There are three common SATA versions: SATA I (150MB/s), SATA II (300MB/s), SATA III (600MB/s)
 +
* There are three common SATA connectors:  Sata Data & Power pair / mSata Connector (aka Mini Sata) / M2 Sata connector (aka NGFF, Next Gen Form Factor)
  
−
===Modern Storage for Retro Computers===
+
'''NVMe'''
−
'''SD to IDE'''
+
* Common after 2018, not IDE, no longer uses ATA protocols
−
Pros
+
* Uses the same M2 connector as NGFF Sata, but uses different pins
−
* SD Cards are cheap
+
* Connects storage directly to PCI-Express
−
* Quick response on small reads and writes
+
* Cannot be used as a boot device without UEFI firmware support and Operating System support
−
*
+
* Microsoft provided Windows 7 drivers, and community created Windows XP drivers
−
'''CF (Compact Flash)'''
+
* Linux GRand Unified Bootloader (GRUB) has support for NVME devices
−
'''Pata DOM (Disk on Module)'''
+
−
'''Sata Drive'''
+
−
'''mSata'''
+
−
'''M2 Sata'''
+
−
'''USB to IDE'''
+
−
'''Sata SSD'''
+
−
'''Sata DOM'''
+
  
 +
== Common BIOS Storage Limitations  ==
 +
* '''July 1994''' - 504/528 MB limit - Original CHS ('''C'''ylinders/'''H'''eads/'''S'''ectors) limit - BIOS before this date rarely support drives over 528MB.  As released, these systems were limited to 1024 Cylinders, 16 Heads, 63 Sectors/Track.  These systems either don't allow larger values to be entered, or were not stable when larger values are configured. 
 +
* '''May 1996''' - 2015 MB limit - INT 13h limit - Some BIOS before this date allow drives up to 2015 MB by updating INT 13h to allow Cylinder values up to 4095.  Head limit remained at 16, Sectors/Track at 63.  Not supported by Microsoft Operating Systems before Win95 SR2.
 +
* '''Phoenix BIOS 4.03 and 4.04''' - 3277 MB limit - The BIOS config screen locks if a drive is configured with a capacity over 3277 MB.
 +
* '''February 1997'''
 +
** 4.2 GB limit - ECHS limit (Extended CHS) - Some BIOS before this date allow a 'Large' mode in the BIOS that produces an alternate geometry by doubling the number of heads and halving the number of cylinders shown to DOS until cylinders <= 1024. The limit for this method for drives that report 16 heads is 4032 MB (C=1024, H = 128, S = 63).
 +
** 7.9 GB limit - Revised ECHS limit.  Other BIOS from this period have a 'Large' mode in the BIOS that presents an alternate geometry using multiples of 15 heads, up to 240 heads.  This method stops working at 7560 MB (C=1024, H=240, S=63)
 +
** 8.4 GB limit - Final ECHS limit - BIOS geometry uses head value from the sequence 16, 32, 64, 128, 255 to present an alternate geometry up to 8.4 GB (C=1024, H=255, S=63). 
 +
* '''August 1999''' - 33.8 GB limit - BIOS before this date often stored the cylinders as a 16 bit value, so they may have issues when a drive reports cylinders > 65535.  Drives from this era often had a jumper to limit cylinders = 65535.
 +
* '''September 2001''' - 137.4 GB limit - BIOS before this date only used ATA-5, which used 28 bits to identify each LBA sector, limiting drive capacity to 137GB.  ATA-6 added an additional 48bit LBA sector field.
 +
* Hard drives that report a CHS geometry of 16383 Cylinders, 16 Heads and 63 Sectors/Track are indicating that they are larger than ECHS geometry translation allows and that LBA or LBA48 should be used
 +
* Hard drives that report an LBA capacity of 0xfffffff are indicating that they are larger than LBA allows and that LBA48 should be used
 +
* Number of Heads had a limit of 255 in MS-DOS so no BIOS was ever enhanced to allow more than 255 Heads.
 +
* Sectors/Track was limited to 63 in early BIOS and was never addressed because increasing the number of allowable Cylinders & switching to LBA fixed most storage issues.
  
 +
== Overcoming Limitations and Incompatibilities ==
 +
* Systems with BIOS limitations can be configured to work with larger drives using one of these three techniques:
 +
** Configure the drive with geometry with Cylinders < 1024, Heads <= 16, and Sectors <= 63 in the BIOS and install [http://vogonsdrivers.com/index.php?catid=19 drive overlay software] that replaces the BIOS routines when the boot sector is loaded
 +
** Upgrade your BIOS using a new BIOS from the motherboard manufacturer or from a third party such as [https://www.vogons.org/viewtopic.php?f=46&t=59146 MR BIOS] that works with your motherboard
 +
** Install an add-in card with an option ROM extension that replaces the BIOS disk routines before BOOT such as [https://www.xtideuniversalbios.org/ XTIDE UniversalBIOS] or [http://vogonsdrivers.com/getfile.php?fileid=1816 SIIG Enhanced BIOS]
 +
* Drive overlay software is usually the easiest option
 +
* "EZ Drive" overlay software usually works with partition types and drive geometries that are compatible with modern operating systems.  This can be helpful if you plan to move storage devices between computers.
 +
* Early LBA drives do not always work correctly with mature LBA controllers.  This can be resolved by configuring the drive mode as 'CHS' or 'Normal' in the BIOS.
 +
* Mature ATA6 storage devices may have trouble negotiating a compatible configuration with some early ATA-3 & ATA-4 controllers.  This can be resolved by disabling ATA33 or UDMA modes in the BIOS.
 +
* PATA speeds faster than UDMA2 (such as ATA-5, Ultra ATA/66, UDMA3, UDMA/66) require a ground or open circuit on IDE pin #34. Usually this can be done with an 80 wire IDE cable, but can also be accomplished by grounding pin #34.
 +
* Some systems with BIOS before 1992 may have trouble accessing very fast storage devices because the expect reuults to stay in the IO bus longer.  Upgrading the BIOS or andding an Option ROM like XTide BIOS usually fixes this.
  
−
===Common Legacy Storage===
+
== Common OS Storage Limitations  ==
−
'''Sata DOM'''
+
* Microsoft DOS (including Windows 95/98) require heads <= 255
 +
* Full support for enhanced INT 13h (Cylinders >= 1024 / disks larger than 8.4GB) arrived with these releases:
 +
** Windows 95 OEM Service Release 2
 +
** Windows 2000
 +
** IBM OS/2 Warp Client version 4.0.14.
 +
** LILO version 21.4
 +
* Windows 98 require a patch for LBA48 support and is limited to 137.4GB LBA drives without it.
 +
* Linux kernels older than 2.0.39 / 2.2.14 / 2.3.21 [https://tldp.org/HOWTO/Large-Disk-HOWTO-12.html require a patch to work with cylinders > 65535].
 +
* Microsoft Operating Systems up through Windows 7 & Server 2008r2 use CHS addressing when booting from a master boot record
  
−
===Rare Legacy Storage===
+
== Common File System / Partitioning Limitations  ==
 +
* DOS with FAT12 allowed for 32MB partitions using the largest cluster size
 +
* DOS with FAT16B allowed for 2GB partitions with the default cluster size
 +
* DOS/Win with FAT32 allowed for 2TB partitions with the default cluster size
 +
 
 +
==Modern Storage for Retro Computers==
 +
 
 +
=== CF (Compact Flash) ===
 +
A popular portable storage format developed in 1994 that is compatible with IDE signaling and ATA protocols.  Standard has not significantly changed since PATA matured.
 +
* CF devices have an extensive family tree and that comes with a wide spectrum of behavior, especially if you have a device with firmware older than 2005
 +
* Devices labeled "industrial" often work best for retro-computing, supporting the best response times & compatibility.
 +
* "Consumer" devices are often optimized for storing large images files and may not perform as well on random access work loads
 +
* CF devices such as a type 1 SD-->CF adapter may require 3.3v power, which requires an IDE adapter with a voltage regulator
 +
* While some CF devices support speeds faster than 133Mhz/UDMA6, those speeds are not accessible from a PC.
 +
 
 +
'''Pros'''
 +
* Still very common and affordable
 +
* Most operate in "true IDE mode", so the devices have native ATA support & use inexpensive pass though adapters
 +
* Portable media that is easy to move between old and new computers
 +
* Rugged form factor that is difficult to damage
 +
'''Cons'''
 +
* New industrial CFs are becoming less common from major brands
 +
* TRIM support on a CF is rare, making CF's less desirable for operating systems that support this feature
 +
* Some CF devices report themselves to be "removable media" when in true IDE mode, which can cause problems when installing Windows 2000 & XP.
 +
* Few CF's support ATA block transfer sizes > 1.  This can cause lower max transfer rates when compared with devices that support block sizes > 1.
 +
* Pin28 issues on old boards.  Some pre 1991 systems experience video corruption with CF's https://www.vogons.org/viewtopic.php?p=1142705
 +
 
 +
[[Media:Example.ogg]]=== SD to IDE Adapters: ===
 +
 
 +
Most SD-->IDE adapters available today are all variations on the Sintechi device with firmware versions 1.2, 1.3 & 1.4.  All perform very similarly.  These devices do not support LBA48, so they may perform erratically when SD cards larger than 128GB are used.  Old & low quality SD devices can limit performance, but generally any < 128GB SD made since 2010 works fine in these devices.
 +
 
 +
'''Pros'''
 +
* SD devices are a very common and affordable storage media
 +
* Usually provides good response on compute work loads (small reads and writes)
 +
* Portable media that is easy to move between old and new computers
 +
'''Cons'''
 +
* Sintechi adapters are limited to High Speed SD (25MB/s) regardless of your SD device's capabilities, limiting the max transfer rates.
 +
* Old & low quality SD devices can limit performance
 +
* Micro SD cards are somewhat fragile compared to other removable media
 +
* TRIM is not supported, making these devices less desirable with operating systems that support TRIM.
 +
 
 +
=== Disk on Module (Dom): ===
 +
'''Pros'''
 +
* Does not require an adapter or cable, sits right on the IDE block for a less cluttered case
 +
* Most DOM devices offer performance on par with the best SD and CF devices
 +
'''Cons'''
 +
* Hardware mod (short pin 34) or Female-Female 40 pin cable necessary to negotiate speeds faster that ATA-4 / UDMA2
 +
* Some DOM devices have issues negotiating ATA protocols with early ATA-2 and ATA-3 controllers
 +
* DOM devices usually have ide latches that firmly connect to the IDE pin-blocks. These latches can stress brittle plastic on old motherboards
 +
* Difficult to connect to a new PC without a Female-Female 40 pin cross over cable
 +
 
 +
=== Sata SSD: ===
 +
'''Pros'''
 +
* Best performance, especially when the device has a write buffer
 +
* Usually supports TRIM and wear leveling
 +
* In 2021, NGFF and M2 devices have affordable pricing for builds between 16GB and 64GB
 +
* Usually responds well to an ATA Secure erase command to reset wear leveling
 +
'''Cons'''
 +
* You may need one or more adapters if you want to connect your device to a 40 pin IDE controller
 +
* More convenient to move between systems than a DOM, but not as easy as SD and CF
 +
* Sata --> Pata converters usually require Sata 1 protocols and some recent Sata devices lack full Sata 1 support.  (e.g. PNY)
 +
 
 +
== Write Amplification and wear leveling: ==
 +
* It isn't clear how any single CF, SD, and DOM devices handles wear leveling without doing a lot of research on that specific device.
 +
* Unclear if any SD devices support trim commands in a way that is usable through an SD-->PATA adapter.
 +
* Few operating systems before 2009 (Windows 7) support TRIM commands, so even if the storage device supports TRIM, it might not be accessible.
 +
* Some retro enthusiasts avoid filling solid state storage devices over 50% as a way reduce the likelihood of write amplification reducing performance.
 +
* For Win98, consider adding "ConservativeSwapfileUsage=1" to your system.ini to reduce unnecessary swap file usage.
 +
* For WinXP, consider adding extra ram to reduce swap file usage.
 +
* It isn't clear how any of these devices will respond to an ATA Secure erase command to reset the device's wear leveling.
 +
 
 +
==Removable Storage Speed Ratings ==
 +
 
 +
=== CF Revisions ===
 +
* CompactFlash 1.0 (1995) supported PIO2 with capacity up to 137 GB.
 +
* CompactFlash 2.0 (2003) added PIO4 followed by DMA 33 in mid-2004.
 +
* CompactFlash 3.0 (2004) added UDMA 66 (UDMA3)
 +
* CompactFlash 4.0 (2006) added IDE Ultra DMA Mode 6 and UDMA 133
 +
* CompactFlash 5.0 (2010) added LBA 48 for drives larger than 137 GB
 +
* CompactFlash 6.0 (November 2010) added UltraDMA Mode 7 (167 MByte/s), ATA-8/ACS-2 sanitize command, and TRIM support
 +
* CFast (2008) Also called CompactFast, is a SATA based standard with a similar form factor that is not interchangeable with PATA CF devices
 +
* XQD card (2011) Is a PCI-E based standard with a similar form factor than is not compatible with either PATA CF or SATA CFast
 +
 
 +
=== SD Speed Ratings ===
 +
 
 +
Early SD speed classes only measured minimum write throughput, which does not align with usual computing work loads.
 +
 
 +
{| class="wikitable" style="text-align: center;"
 +
|+ Comparison of SD card Speed Class ratings
 +
! scope="col" rowspan="2" | Minimum sequential
 +
writing speed
 +
!
 +
! scope="col" colspan="3" | Speed Class
 +
 
 +
|-
 +
!Suggested max. bitrate
 +
! scope="col" | Speed Class
 +
! scope="col" | UHS Speed Class
 +
! scope="col" | Video Speed Class
 +
|-
 +
! scope="row" | 2 MB/s
 +
! 15 Mbit/s
 +
| Class 2 (C2)
 +
| N/A
 +
| N/A
 +
|-
 +
! scope="row" | 4 MB/s
 +
! 30 Mbit/s
 +
| Class 4 (C4)
 +
| N/A
 +
| N/A
 +
|-
 +
! scope="row" | 6 MB/s
 +
! 45 Mbit/s
 +
| Class 6 (C6)
 +
| N/A
 +
| Class 6 (V6)
 +
|-
 +
! scope="row" | 10 MB/s
 +
! 75 Mbit/s
 +
| Class 10 (C10)
 +
| Class 1 (U1)
 +
| Class 10 (V10)
 +
|-
 +
! scope="row" | 30 MB/s
 +
! 220 Mbit/s
 +
|
 +
| Class 3 (U3)
 +
| Class 30 (V30)
 +
|-
 +
! scope="row" | 60 MB/s
 +
! 460 Mbit/s
 +
|
 +
|
 +
| Class 60 (V60)
 +
|-
 +
! scope="row" | 90 MB/s
 +
! 700 Mbit/s
 +
|
 +
|
 +
| Class 90 (V90)
 +
|}
 +
 
 +
Application Performance Class is defined in SD Specification 5.1 and 6.0 which mandates a minimum IOPS for reading and writing, which more closely aligns with computing work loads.
 +
 
 +
{| class="wikitable"
 +
|+ Comparison of SD card Application Performance Class ratings
 +
! rowspan="2" scope="col" | Name
 +
! colspan="2" scope="col" | Minimum random IOPS
 +
! rowspan="2" scope="col" | Minimum sustained sequential writing
 +
|-
 +
!Read
 +
!Write
 +
|-
 +
! scope="row" | Application Performance Class 1 (A1)
 +
| 1500 IOPS
 +
| 500 IOPS
 +
| rowspan="2" |10 MB/s
 +
|-
 +
! scope="row" |Application Performance Class 2 (A2)
 +
|4000 IOPS
 +
|2000 IOPS
 +
|}

Latest revision as of 08:29, 5 July 2024

Common IDE Categories

Early IDE (ATA-0 & ATA-1)

  • Commonly used with 286, 386, and early 486 computers
  • IDE for 16bit ISA controllers, connects IDE storage devices directly to the ISA/AT bus
  • Storage transfer rates depend on the clock speed of the ISA bus
  • Most devices use PIO transfers.
  • DMA support is extremely rare because it was not faster than PIO on an ISA bus.

XT-IDE

  • This is a card that allows 16 bit IDE devices to work in an 8 bit slot
  • Also used to describe storage devices that support a rarely used 8 bit transfer protocol
  • Also used as shorthand for XtIDE Universal BIOS, which is an ISA Option ROM for PC's and not drive controller or storage device.

EIDE, Fast ATA (ATA-2 & ATA-3 )

  • Common between 1994-1997: Late Socket 3, Socket 4,5,6, early Socket 7
  • PIO modes transfer rates up to 16Mhz available on some VLB controllers and most PCI controllers
  • Multiword DMA transfer rates to 16Mhz available on a few VLB controllers and most PCI controllers.
  • Controllers from this period were often paired with BIOS support for storage >512MB using ECHS addressing
  • VLB controllers from this period sometimes have jumpers that allow transfer speed to be configured without a driver.
  • VLB controllers from this period sometimes have a jumper to allow the use of IO Ready signaling to increase the efficiency of PIO modes, but this requires driver support.

ATA-4, UDMA/33, Ultra ATA/33

  • Common between 1998-1999: Pentium II & III, Super Socket 7, Early Athlon
  • UDMA2 operated at 33MHz for significant performance increases over previous generations
  • 80 connector IDE cables recommended but not required
  • UDMA bus master transfer modes significantly improve performance in multitasking operating systems
  • Few BIOS directly allow UDMA transfers in DOS without a UMDA driver in config.sys
  • Windows 98 requires users to change a setting in Device manager to enable UDMA Bus mastering
  • Some Award BIOS from this period require a patch to allow UDMA operation in Win98. https://www.vogons.org/viewtopic.php?p=1272522#p1272522
  • Controllers from this period were often paired with BIOS support that support >8.4GB using LBA28 addressing

Mature PATA (ATA-5, ATA-6, ATA-7: Ultra ATA-66, ATA-100 & ATA-133)

  • Common between 2000-2010: Later Pentium 3 & Athlon through Core2
  • UDMA4 & UDMA5 & UDMA6 require an 80 connector IDE cable. An 80 connector can be simulated by grounding Pin #34 on the storage device. https://www.vogons.org/viewtopic.php?t=77881
  • UDMA5 & UDMA6 rarely show significant performance improvement over UDMA4 unless paired with the fastest PATA storage devices and a south bridge that uses >= Intel ICH5, VIA Enhanced-Vlink or AMD HyperTransport.
  • Controllers from this period were often paired with BIOS support for storage >128GB using LBA48

More information about PATA ATA is available here: https://en.wikipedia.org/wiki/Parallel_ATA

SATA

  • Common after 2004
  • Early SATA devices were often PATA devices with a built in SATA bridge
  • Bridges are commonly available today to connect PATA devices to SATA controllers & vice versa.
  • Most native SATA devices support ACHI features such as NCQ, TRIM, better power management and hot swapping, but these features can be difficult or impossible to access when the device is attached through a PATA bridge.
  • There are three common SATA versions: SATA I (150MB/s), SATA II (300MB/s), SATA III (600MB/s)
  • There are three common SATA connectors: Sata Data & Power pair / mSata Connector (aka Mini Sata) / M2 Sata connector (aka NGFF, Next Gen Form Factor)

NVMe

  • Common after 2018, not IDE, no longer uses ATA protocols
  • Uses the same M2 connector as NGFF Sata, but uses different pins
  • Connects storage directly to PCI-Express
  • Cannot be used as a boot device without UEFI firmware support and Operating System support
  • Microsoft provided Windows 7 drivers, and community created Windows XP drivers
  • Linux GRand Unified Bootloader (GRUB) has support for NVME devices

Common BIOS Storage Limitations

  • July 1994 - 504/528 MB limit - Original CHS (Cylinders/Heads/Sectors) limit - BIOS before this date rarely support drives over 528MB. As released, these systems were limited to 1024 Cylinders, 16 Heads, 63 Sectors/Track. These systems either don't allow larger values to be entered, or were not stable when larger values are configured.
  • May 1996 - 2015 MB limit - INT 13h limit - Some BIOS before this date allow drives up to 2015 MB by updating INT 13h to allow Cylinder values up to 4095. Head limit remained at 16, Sectors/Track at 63. Not supported by Microsoft Operating Systems before Win95 SR2.
  • Phoenix BIOS 4.03 and 4.04 - 3277 MB limit - The BIOS config screen locks if a drive is configured with a capacity over 3277 MB.
  • February 1997
    • 4.2 GB limit - ECHS limit (Extended CHS) - Some BIOS before this date allow a 'Large' mode in the BIOS that produces an alternate geometry by doubling the number of heads and halving the number of cylinders shown to DOS until cylinders <= 1024. The limit for this method for drives that report 16 heads is 4032 MB (C=1024, H = 128, S = 63).
    • 7.9 GB limit - Revised ECHS limit. Other BIOS from this period have a 'Large' mode in the BIOS that presents an alternate geometry using multiples of 15 heads, up to 240 heads. This method stops working at 7560 MB (C=1024, H=240, S=63)
    • 8.4 GB limit - Final ECHS limit - BIOS geometry uses head value from the sequence 16, 32, 64, 128, 255 to present an alternate geometry up to 8.4 GB (C=1024, H=255, S=63).
  • August 1999 - 33.8 GB limit - BIOS before this date often stored the cylinders as a 16 bit value, so they may have issues when a drive reports cylinders > 65535. Drives from this era often had a jumper to limit cylinders = 65535.
  • September 2001 - 137.4 GB limit - BIOS before this date only used ATA-5, which used 28 bits to identify each LBA sector, limiting drive capacity to 137GB. ATA-6 added an additional 48bit LBA sector field.
  • Hard drives that report a CHS geometry of 16383 Cylinders, 16 Heads and 63 Sectors/Track are indicating that they are larger than ECHS geometry translation allows and that LBA or LBA48 should be used
  • Hard drives that report an LBA capacity of 0xfffffff are indicating that they are larger than LBA allows and that LBA48 should be used
  • Number of Heads had a limit of 255 in MS-DOS so no BIOS was ever enhanced to allow more than 255 Heads.
  • Sectors/Track was limited to 63 in early BIOS and was never addressed because increasing the number of allowable Cylinders & switching to LBA fixed most storage issues.

Overcoming Limitations and Incompatibilities

  • Systems with BIOS limitations can be configured to work with larger drives using one of these three techniques:
    • Configure the drive with geometry with Cylinders < 1024, Heads <= 16, and Sectors <= 63 in the BIOS and install drive overlay software that replaces the BIOS routines when the boot sector is loaded
    • Upgrade your BIOS using a new BIOS from the motherboard manufacturer or from a third party such as MR BIOS that works with your motherboard
    • Install an add-in card with an option ROM extension that replaces the BIOS disk routines before BOOT such as XTIDE UniversalBIOS or SIIG Enhanced BIOS
  • Drive overlay software is usually the easiest option
  • "EZ Drive" overlay software usually works with partition types and drive geometries that are compatible with modern operating systems. This can be helpful if you plan to move storage devices between computers.
  • Early LBA drives do not always work correctly with mature LBA controllers. This can be resolved by configuring the drive mode as 'CHS' or 'Normal' in the BIOS.
  • Mature ATA6 storage devices may have trouble negotiating a compatible configuration with some early ATA-3 & ATA-4 controllers. This can be resolved by disabling ATA33 or UDMA modes in the BIOS.
  • PATA speeds faster than UDMA2 (such as ATA-5, Ultra ATA/66, UDMA3, UDMA/66) require a ground or open circuit on IDE pin #34. Usually this can be done with an 80 wire IDE cable, but can also be accomplished by grounding pin #34.
  • Some systems with BIOS before 1992 may have trouble accessing very fast storage devices because the expect reuults to stay in the IO bus longer. Upgrading the BIOS or andding an Option ROM like XTide BIOS usually fixes this.

Common OS Storage Limitations

  • Microsoft DOS (including Windows 95/98) require heads <= 255
  • Full support for enhanced INT 13h (Cylinders >= 1024 / disks larger than 8.4GB) arrived with these releases:
    • Windows 95 OEM Service Release 2
    • Windows 2000
    • IBM OS/2 Warp Client version 4.0.14.
    • LILO version 21.4
  • Windows 98 require a patch for LBA48 support and is limited to 137.4GB LBA drives without it.
  • Linux kernels older than 2.0.39 / 2.2.14 / 2.3.21 require a patch to work with cylinders > 65535.
  • Microsoft Operating Systems up through Windows 7 & Server 2008r2 use CHS addressing when booting from a master boot record

Common File System / Partitioning Limitations

  • DOS with FAT12 allowed for 32MB partitions using the largest cluster size
  • DOS with FAT16B allowed for 2GB partitions with the default cluster size
  • DOS/Win with FAT32 allowed for 2TB partitions with the default cluster size

Modern Storage for Retro Computers

CF (Compact Flash)

A popular portable storage format developed in 1994 that is compatible with IDE signaling and ATA protocols. Standard has not significantly changed since PATA matured.

  • CF devices have an extensive family tree and that comes with a wide spectrum of behavior, especially if you have a device with firmware older than 2005
  • Devices labeled "industrial" often work best for retro-computing, supporting the best response times & compatibility.
  • "Consumer" devices are often optimized for storing large images files and may not perform as well on random access work loads
  • CF devices such as a type 1 SD-->CF adapter may require 3.3v power, which requires an IDE adapter with a voltage regulator
  • While some CF devices support speeds faster than 133Mhz/UDMA6, those speeds are not accessible from a PC.

Pros

  • Still very common and affordable
  • Most operate in "true IDE mode", so the devices have native ATA support & use inexpensive pass though adapters
  • Portable media that is easy to move between old and new computers
  • Rugged form factor that is difficult to damage

Cons

  • New industrial CFs are becoming less common from major brands
  • TRIM support on a CF is rare, making CF's less desirable for operating systems that support this feature
  • Some CF devices report themselves to be "removable media" when in true IDE mode, which can cause problems when installing Windows 2000 & XP.
  • Few CF's support ATA block transfer sizes > 1. This can cause lower max transfer rates when compared with devices that support block sizes > 1.
  • Pin28 issues on old boards. Some pre 1991 systems experience video corruption with CF's https://www.vogons.org/viewtopic.php?p=1142705

Media:Example.ogg=== SD to IDE Adapters: ===

Most SD-->IDE adapters available today are all variations on the Sintechi device with firmware versions 1.2, 1.3 & 1.4. All perform very similarly. These devices do not support LBA48, so they may perform erratically when SD cards larger than 128GB are used. Old & low quality SD devices can limit performance, but generally any < 128GB SD made since 2010 works fine in these devices.

Pros

  • SD devices are a very common and affordable storage media
  • Usually provides good response on compute work loads (small reads and writes)
  • Portable media that is easy to move between old and new computers

Cons

  • Sintechi adapters are limited to High Speed SD (25MB/s) regardless of your SD device's capabilities, limiting the max transfer rates.
  • Old & low quality SD devices can limit performance
  • Micro SD cards are somewhat fragile compared to other removable media
  • TRIM is not supported, making these devices less desirable with operating systems that support TRIM.

Disk on Module (Dom):

Pros

  • Does not require an adapter or cable, sits right on the IDE block for a less cluttered case
  • Most DOM devices offer performance on par with the best SD and CF devices

Cons

  • Hardware mod (short pin 34) or Female-Female 40 pin cable necessary to negotiate speeds faster that ATA-4 / UDMA2
  • Some DOM devices have issues negotiating ATA protocols with early ATA-2 and ATA-3 controllers
  • DOM devices usually have ide latches that firmly connect to the IDE pin-blocks. These latches can stress brittle plastic on old motherboards
  • Difficult to connect to a new PC without a Female-Female 40 pin cross over cable

Sata SSD:

Pros

  • Best performance, especially when the device has a write buffer
  • Usually supports TRIM and wear leveling
  • In 2021, NGFF and M2 devices have affordable pricing for builds between 16GB and 64GB
  • Usually responds well to an ATA Secure erase command to reset wear leveling

Cons

  • You may need one or more adapters if you want to connect your device to a 40 pin IDE controller
  • More convenient to move between systems than a DOM, but not as easy as SD and CF
  • Sata --> Pata converters usually require Sata 1 protocols and some recent Sata devices lack full Sata 1 support. (e.g. PNY)

Write Amplification and wear leveling:

  • It isn't clear how any single CF, SD, and DOM devices handles wear leveling without doing a lot of research on that specific device.
  • Unclear if any SD devices support trim commands in a way that is usable through an SD-->PATA adapter.
  • Few operating systems before 2009 (Windows 7) support TRIM commands, so even if the storage device supports TRIM, it might not be accessible.
  • Some retro enthusiasts avoid filling solid state storage devices over 50% as a way reduce the likelihood of write amplification reducing performance.
  • For Win98, consider adding "ConservativeSwapfileUsage=1" to your system.ini to reduce unnecessary swap file usage.
  • For WinXP, consider adding extra ram to reduce swap file usage.
  • It isn't clear how any of these devices will respond to an ATA Secure erase command to reset the device's wear leveling.

Removable Storage Speed Ratings

CF Revisions

  • CompactFlash 1.0 (1995) supported PIO2 with capacity up to 137 GB.
  • CompactFlash 2.0 (2003) added PIO4 followed by DMA 33 in mid-2004.
  • CompactFlash 3.0 (2004) added UDMA 66 (UDMA3)
  • CompactFlash 4.0 (2006) added IDE Ultra DMA Mode 6 and UDMA 133
  • CompactFlash 5.0 (2010) added LBA 48 for drives larger than 137 GB
  • CompactFlash 6.0 (November 2010) added UltraDMA Mode 7 (167 MByte/s), ATA-8/ACS-2 sanitize command, and TRIM support
  • CFast (2008) Also called CompactFast, is a SATA based standard with a similar form factor that is not interchangeable with PATA CF devices
  • XQD card (2011) Is a PCI-E based standard with a similar form factor than is not compatible with either PATA CF or SATA CFast

SD Speed Ratings

Early SD speed classes only measured minimum write throughput, which does not align with usual computing work loads.

Comparison of SD card Speed Class ratings
Minimum sequential

writing speed

Speed Class
Suggested max. bitrate Speed Class UHS Speed Class Video Speed Class
2 MB/s 15 Mbit/s Class 2 (C2) N/A N/A
4 MB/s 30 Mbit/s Class 4 (C4) N/A N/A
6 MB/s 45 Mbit/s Class 6 (C6) N/A Class 6 (V6)
10 MB/s 75 Mbit/s Class 10 (C10) Class 1 (U1) Class 10 (V10)
30 MB/s 220 Mbit/s Class 3 (U3) Class 30 (V30)
60 MB/s 460 Mbit/s Class 60 (V60)
90 MB/s 700 Mbit/s Class 90 (V90)

Application Performance Class is defined in SD Specification 5.1 and 6.0 which mandates a minimum IOPS for reading and writing, which more closely aligns with computing work loads.

Comparison of SD card Application Performance Class ratings
Name Minimum random IOPS Minimum sustained sequential writing
Read Write
Application Performance Class 1 (A1) 1500 IOPS 500 IOPS 10 MB/s
Application Performance Class 2 (A2) 4000 IOPS 2000 IOPS
Retrieved from "https://www.vogonswiki.com/index.php?title=Storage&oldid=4941"