Difference between revisions of "DOS memory management"
(→Expanded memory) |
(→Additional Remarks) |
||
| (14 intermediate revisions by the same user not shown) | |||
| Line 10: | Line 10: | ||
=== Upper memory === | === Upper memory === | ||
| − | The upper memory area is memory in the range between 640 kb and 1 Mb. By default there is no RAM in this range as it is reserved for use with hardware that is able to map own memory to this range. Usually present in this region is a part of the graphics cards RAM and the BIOS ROMs of the graphics card and mainboard. Additional hardware like mass storage controllers, network adapters... can use | + | The '''upper memory area''' ('''UMA''') is memory in the range between 640 kb and 1 Mb. By default there is no RAM in this range as it is reserved for use with hardware that is able to map own memory to this range. Usually present in this region is a part of the graphics cards RAM and the BIOS ROMs of the graphics card and mainboard. Additional hardware like mass storage controllers, network adapters... can use parts of the UMA for own BIOS ROMs or buffer RAM. The Extended Memory Specification (XMS) allows to map blocks of mainboard RAM in the remaining UMA memory range. These blocks are called '''Upper Memory Blocks''' (UMB) and are treated from DOS as High Memory. The default memory management driver that enables this mapping through XMS memory is EMM386.EXE. It has to be loaded in config.sys as second driver after HIMEM.SYS. The driver tries to detect common unused blocks and maps XMS memory there creating Upper Memory Blocks. |
| − | TSR-programs | + | By specifying DOS=UMB in config.sys MS-DOS allocates all UMBs through XMS and takes over memory management of them. this allows TSR-programs to be loaded to UMBs (in High Memory) in config.sys by using ''DEVICEHIGH=driver.sys'' or in autoexec.bat with the loadhigh statement, like ''LH driver.com''. If no contiguous free Upper Memory Block is available the driver will be loaded to Conventional Memory. Since UMA memory is managed in blocks the amount of free Upper Memory is usually larger than the largest contiguous free block. The external program MEM with parameter /C reports additionally the UMA RAM status. |
=== High memory area === | === High memory area === | ||
| − | The High Memory Area (HMA) are the 64 kb directly above 1 Mb. To use HMA it is required to load HIMEM.SYS in config.sys | + | The High Memory Area ('''HMA''') are the 64 kb directly above 1 Mb and are part of XMS. To use HMA it is required to load HIMEM.SYS in config.sys. By specifying the line '''DOS=HIGH''' in config.sys DOS is asked to load parts to HMA. Directly after an XMS driver is loaded in config.sys with DEVICE= and the HMA is still unoccupied DOS will move parts of code there. If DOS=HIGH was specified but loading to HMA fails ''HMA not available/loading DOS low'' is reported. |
| + | The external program MEM reports if DOS is using the HMA. | ||
=== Expanded memory === | === Expanded memory === | ||
| − | The Expanded Memory Specification (EMS) is a standard developed by Lotus, Intel and Microsoft. Expanded Memory can be either memory on an memory expansion card or a part of the main memory. The specification describes that this memory can be used by mapping a 64 kB large part to the Upper Memory Area between 640 kB and 1 Mb. The 64 kB region within the Upper Memory Area is referred to as EMS Page Frame. The latest EMS 4.0 allowed to use up to 32 Mb as expanded memory. The most commonly used memory management driver implementing EMS is EMM386.EXE that has to be loaded in config.sys as second driver after HIMEM.SYS. EMM386 emulates expanded memory by using main memory beyond the High Memory Area. To achieve this it has to switch the CPU to Protected Mode. | + | The '''Expanded Memory Specification''' ('''EMS''') is a standard developed by Lotus, Intel and Microsoft. Expanded Memory can be either memory on an memory expansion card or a part of the main memory. The specification describes that this memory can be used by mapping a 64 kB large part to the Upper Memory Area between 640 kB and 1 Mb. The 64 kB region within the Upper Memory Area is referred to as EMS Page Frame. The latest EMS 4.0 allowed to use up to 32 Mb as expanded memory. The most commonly used memory management driver implementing EMS is EMM386.EXE that has to be loaded in config.sys as second driver after HIMEM.SYS. EMM386 emulates expanded memory by using main memory beyond the High Memory Area. To achieve this it has to switch the CPU to Protected Mode. The amount of free main memory beyond 1 MB can be shared between EMS and XMS by adding the parameter AUTO to EMM386.EXE. |
In the case no EMS is required this feature can be deactivated by using the parameter NOEMS. This frees 64 kB in the Upper Memory Area for loading TSR-programs. | In the case no EMS is required this feature can be deactivated by using the parameter NOEMS. This frees 64 kB in the Upper Memory Area for loading TSR-programs. | ||
| + | |||
| + | Versions: | ||
| + | |||
| + | EMS 3.0: max. 4 Mb | ||
| + | |||
| + | EMS 3.2: max. 8 Mb | ||
| + | |||
| + | EMS 4.0: max. 32 Mb | ||
=== Extended memory === | === Extended memory === | ||
| − | XMS | + | The '''Extended Memory Specification''' ('''XMS''') describes an application interface that allows to copy memory between conventional and extended memory. Extended Memory is all main memory beyond 1 Mb. The most common memory management driver for this functionality is HIMEM.SYS that has to be loaded as first driver in config.sys. Accessing memory above the High Memory Area requires switching the CPU to Protected Mode, thus HIMEM.SYS requires at least a 286 CPU. |
| − | ( | + | |
| + | Versions: | ||
| + | |||
| + | XMS 2.0: max. 64 Mb | ||
| + | |||
| + | XMS 3.0: max. 4 Gb | ||
| + | |||
| + | |||
| + | == Additional Remarks == | ||
| + | |||
| + | While UMBs are a optional feature of XMS HIMEM.SYS does not support UMB. On bare MS-DOS this requires to load EMM386.EXE. | ||
| + | |||
| + | The DOS=HIGH and DOS=UMB statements in config.sys can be combined by writing DOS=HIGH,UMB. Specifying DOS=LOW can be used to prevent DOS from using the HMA. | ||
| + | |||
| + | Shadow-RAM is the RAM below the reserved UMA. Depending on chipset this RAM can be mapped in again in specific UMA memory ranges. A common use of Shadow-RAM is copying the BIOS ROMs to the Shadow-RAM below and mapping out the ROM afterwards. As the BIOS code is executed now from RAM instead of slow ROM the BIOS calls get accelerated. This functionality can usually enabled by BIOS (''Shadow System BIOS'' and ''Shadow Video BIOS'') or as parameter of a memory manager. Also the UMBPCI driver uses this feature to create UMBs through Shadow-RAM. | ||
| + | |||
| + | Architectural reasons force the memory for DMA transfers of ISA cards to be reserved below 1 Mb. This is usually of 16 kB size and reserved in UMA. | ||
== Memory management == | == Memory management == | ||
| Line 31: | Line 56: | ||
On 286 CPUs special chipset support is required to enable UMBs. Rather well supported is the NEAT chipset. The HIMEM.SYS driver takes control of the A20-gate, makes HMA available and installs a XMS API. | On 286 CPUs special chipset support is required to enable UMBs. Rather well supported is the NEAT chipset. The HIMEM.SYS driver takes control of the A20-gate, makes HMA available and installs a XMS API. | ||
| − | It is also possible to emulate a EMS page frame with the EMM286 driver. | + | It is also possible to emulate a EMS page frame with the EMM286 driver. Some chipsets have jumpers to set XMS and EMS assignment of the main memory. |
386 and later CPUs feature an protected mode extension called Virtual 8086 mode that allows mimicking a 8086 memory layout for applications. With MS-DOS 5 a second memory manager was included called EMM386.EXE that uses the advanced features of 386 CPUs. It is loaded after HIMEM.SYS and enables UMBs and EMS emulation. This allows to load TSRs to UMBs freeing conventional RAM. The corresponding load statement in config.sys is DEVICEHIGH= and in autoexec.bat LOADHIGH which can be shortened to LH. EMM386 switches the CPU to Virtual 8086 mode. | 386 and later CPUs feature an protected mode extension called Virtual 8086 mode that allows mimicking a 8086 memory layout for applications. With MS-DOS 5 a second memory manager was included called EMM386.EXE that uses the advanced features of 386 CPUs. It is loaded after HIMEM.SYS and enables UMBs and EMS emulation. This allows to load TSRs to UMBs freeing conventional RAM. The corresponding load statement in config.sys is DEVICEHIGH= and in autoexec.bat LOADHIGH which can be shortened to LH. EMM386 switches the CPU to Virtual 8086 mode. | ||
| Line 40: | Line 65: | ||
Mainboard BIOS, graphic card BIOS and also additional cards as mass storage controllers or network cards use the memory area between 640K and 1 MB sharing space with UMBs. Especially newer graphic cards and mainboards are not designed with DOS in mind and are delivered with large BIOSes (> 32 kB). This can reduce available UMBs considerably. A classic DOS system has about 128 kB to 192 kB free UMBs for TSRs. Using EMS emulation requires a 64 kB area between 640K and 1 MB where the page frame of a virtual EMS card is mapped. If not required EMS emulation can be disabled freeing 64 kB of UMBs. | Mainboard BIOS, graphic card BIOS and also additional cards as mass storage controllers or network cards use the memory area between 640K and 1 MB sharing space with UMBs. Especially newer graphic cards and mainboards are not designed with DOS in mind and are delivered with large BIOSes (> 32 kB). This can reduce available UMBs considerably. A classic DOS system has about 128 kB to 192 kB free UMBs for TSRs. Using EMS emulation requires a 64 kB area between 640K and 1 MB where the page frame of a virtual EMS card is mapped. If not required EMS emulation can be disabled freeing 64 kB of UMBs. | ||
| + | Another way is to use a utility that changes the start up files such that the amount of free conventional memory is maximal (e.g. MS-DOS Memmaker, QEMM Optimize). Typically these programs run several analysis reboots to gather information about the drivers and memory layout. In the end a optimized configuration is presented. The analysis depth and the final quality of the presented new boot configuration of the programs varies. | ||
(todo: expand) | (todo: expand) | ||
== Software == | == Software == | ||
| − | ( | + | '''HIMEM.SYS''': enables XMS (except UMBs) with 286 up to 15 Mb, with 386/486 up to 1023 Mb |
| + | |||
| + | Versions: | ||
| + | |||
| + | MS-DOS 5.0 : 2.77, max. 15 Mb XMS | ||
| + | |||
| + | MS-DOS 6.0 : 3.07, recognizes max. 4 Gb, 1023 MB XMS useable. | ||
| + | |||
| + | MS-DOS 6.2 6.21 6.22 : 3.10, recognizes max. 4 Gb, 1023 MB XMS useable. | ||
| + | |||
| + | '''EMM386.EXE''': introduced with MS-DOS 5.0, uses XMS to create UMBs in UMA, uses XMS to create EMS | ||
| + | |||
| + | '''MEMMAKER.EXE''': introduced with MS-DOS 6.0, utility to optimize free conventional memory | ||
| + | |||
| + | |||
| + | '''QEMM386''': a complete package of memory management driver and utilities, last version 9.0 / QEMM97 | ||
| + | |||
| + | '''QEMM386.SYS''' memory management driver implementing XMS, EMS, UMA, VCPI and DPMI, enhanced features include support for up to 256 MB RAM, QuickBoot, Stealth, dynamically load drivers from command line and specific support for popular drivers as DoubleSpace, Stacker. Additional utilities include | ||
| + | |||
| + | '''Optimize''' a powerful tool to optimize free conventional memory | ||
| + | |||
| + | '''DOS-UP''' for relocating DOS to high memory | ||
| + | |||
| + | '''Manifest''' a system information tool with focus on memory layout | ||
| + | |||
| + | '''QDPMI''' a DPMI 0.9 server | ||
| + | |||
| + | '''MagnaRAM''' a memory compression utility replacing a portion of Windows 3.1 virtual memory subsystem. | ||
| + | |||
| + | |||
| + | '''UMBPCI''' Upper memory driver, partial replacement for EMM386.EXE. Uses Shadow-RAM instead of extended memory to create UMBs. Functionality is chipset dependent, see [http://www.uwe-sieber.de/umbpci_e.html UMBPCI homepage] for a list of supported chipsets. Also be aware of issues with ISA-DMA using the Shadow-RAM method that could lead to data corruption if not setup properly. This driver leaves the CPU in Real mode. | ||
| + | |||
| + | [http://support.microsoft.com/kb/95555 Overview of Memory-Management Functionality in MS-DOS] | ||
| + | |||
| + | [http://en.wikipedia.org/wiki/QEMM#DOS_equivalents Overview about features of different memory managers] | ||
Latest revision as of 01:17, 12 April 2013
With modern operating systems, all memory management you need to do is installing enough RAM. And that's it. The OS determines how to use it, and you usually don't need to worry about it at all. In DOS hardware and memory access is closely related to the standards defined by the x86 architecture. In DOS memory is split into different regions and there are several standards as EMS and XMS for accessing memory. To use the available RAM effectively in DOS you have to know how the x86 architecture splits up RAM, what these different regions are for. It is also helpful to know the most common memory management drivers. Optimizing your system to attain enough free memory for your programs is called memory management.
This page will try to explain, in simple language, all the technical terms you will encounter in DOS memory management, and provide you with practical information to get your favorite games and programs running. If you really want to understand how this all works, see the Wikipedia article on DOS memory management, or use your web search engine of choice.
Contents
Types of memory
Conventional memory
Conventional memory or base memory is the memory range between 0 kb and 640 kb. Programs are loaded to this memory range. The available free memory can be lower as some drivers have to leave a part of their code in memory to handle e.g. hardware. This is called Terminate and Stay Resident (TSR). By default such drivers are placed in conventional memory. The amount of free conventional memory is reported by CHKDSK or MEM.
Upper memory
The upper memory area (UMA) is memory in the range between 640 kb and 1 Mb. By default there is no RAM in this range as it is reserved for use with hardware that is able to map own memory to this range. Usually present in this region is a part of the graphics cards RAM and the BIOS ROMs of the graphics card and mainboard. Additional hardware like mass storage controllers, network adapters... can use parts of the UMA for own BIOS ROMs or buffer RAM. The Extended Memory Specification (XMS) allows to map blocks of mainboard RAM in the remaining UMA memory range. These blocks are called Upper Memory Blocks (UMB) and are treated from DOS as High Memory. The default memory management driver that enables this mapping through XMS memory is EMM386.EXE. It has to be loaded in config.sys as second driver after HIMEM.SYS. The driver tries to detect common unused blocks and maps XMS memory there creating Upper Memory Blocks. By specifying DOS=UMB in config.sys MS-DOS allocates all UMBs through XMS and takes over memory management of them. this allows TSR-programs to be loaded to UMBs (in High Memory) in config.sys by using DEVICEHIGH=driver.sys or in autoexec.bat with the loadhigh statement, like LH driver.com. If no contiguous free Upper Memory Block is available the driver will be loaded to Conventional Memory. Since UMA memory is managed in blocks the amount of free Upper Memory is usually larger than the largest contiguous free block. The external program MEM with parameter /C reports additionally the UMA RAM status.
High memory area
The High Memory Area (HMA) are the 64 kb directly above 1 Mb and are part of XMS. To use HMA it is required to load HIMEM.SYS in config.sys. By specifying the line DOS=HIGH in config.sys DOS is asked to load parts to HMA. Directly after an XMS driver is loaded in config.sys with DEVICE= and the HMA is still unoccupied DOS will move parts of code there. If DOS=HIGH was specified but loading to HMA fails HMA not available/loading DOS low is reported. The external program MEM reports if DOS is using the HMA.
Expanded memory
The Expanded Memory Specification (EMS) is a standard developed by Lotus, Intel and Microsoft. Expanded Memory can be either memory on an memory expansion card or a part of the main memory. The specification describes that this memory can be used by mapping a 64 kB large part to the Upper Memory Area between 640 kB and 1 Mb. The 64 kB region within the Upper Memory Area is referred to as EMS Page Frame. The latest EMS 4.0 allowed to use up to 32 Mb as expanded memory. The most commonly used memory management driver implementing EMS is EMM386.EXE that has to be loaded in config.sys as second driver after HIMEM.SYS. EMM386 emulates expanded memory by using main memory beyond the High Memory Area. To achieve this it has to switch the CPU to Protected Mode. The amount of free main memory beyond 1 MB can be shared between EMS and XMS by adding the parameter AUTO to EMM386.EXE. In the case no EMS is required this feature can be deactivated by using the parameter NOEMS. This frees 64 kB in the Upper Memory Area for loading TSR-programs.
Versions:
EMS 3.0: max. 4 Mb
EMS 3.2: max. 8 Mb
EMS 4.0: max. 32 Mb
Extended memory
The Extended Memory Specification (XMS) describes an application interface that allows to copy memory between conventional and extended memory. Extended Memory is all main memory beyond 1 Mb. The most common memory management driver for this functionality is HIMEM.SYS that has to be loaded as first driver in config.sys. Accessing memory above the High Memory Area requires switching the CPU to Protected Mode, thus HIMEM.SYS requires at least a 286 CPU.
Versions:
XMS 2.0: max. 64 Mb
XMS 3.0: max. 4 Gb
Additional Remarks
While UMBs are a optional feature of XMS HIMEM.SYS does not support UMB. On bare MS-DOS this requires to load EMM386.EXE.
The DOS=HIGH and DOS=UMB statements in config.sys can be combined by writing DOS=HIGH,UMB. Specifying DOS=LOW can be used to prevent DOS from using the HMA.
Shadow-RAM is the RAM below the reserved UMA. Depending on chipset this RAM can be mapped in again in specific UMA memory ranges. A common use of Shadow-RAM is copying the BIOS ROMs to the Shadow-RAM below and mapping out the ROM afterwards. As the BIOS code is executed now from RAM instead of slow ROM the BIOS calls get accelerated. This functionality can usually enabled by BIOS (Shadow System BIOS and Shadow Video BIOS) or as parameter of a memory manager. Also the UMBPCI driver uses this feature to create UMBs through Shadow-RAM.
Architectural reasons force the memory for DMA transfers of ISA cards to be reserved below 1 Mb. This is usually of 16 kB size and reserved in UMA.
Memory management
Getting more conventional memory
A challenging aspect of DOS memory management can be to free enough conventional memory. Often games require a high amount of free conventional memory and sometimes also free EMS or XMS. To achieve this you have to use memory managers that allow e.g. enabling of UMBs and installation of APIs for XMS and EMS access. The availability of memory types depends on CPU and chipset.
On 286 CPUs special chipset support is required to enable UMBs. Rather well supported is the NEAT chipset. The HIMEM.SYS driver takes control of the A20-gate, makes HMA available and installs a XMS API. It is also possible to emulate a EMS page frame with the EMM286 driver. Some chipsets have jumpers to set XMS and EMS assignment of the main memory.
386 and later CPUs feature an protected mode extension called Virtual 8086 mode that allows mimicking a 8086 memory layout for applications. With MS-DOS 5 a second memory manager was included called EMM386.EXE that uses the advanced features of 386 CPUs. It is loaded after HIMEM.SYS and enables UMBs and EMS emulation. This allows to load TSRs to UMBs freeing conventional RAM. The corresponding load statement in config.sys is DEVICEHIGH= and in autoexec.bat LOADHIGH which can be shortened to LH. EMM386 switches the CPU to Virtual 8086 mode.
(todo UMBPCI)
Generally memory managers must be loaded first. As rule of thumb TSRs taking more memory should be loaded before small TSRs to prevent memory fragmentation. Some drivers require a specific load order e.g. SMARTDRV.EXE after MSCDEX.EXE if the CD-ROM should be read cached. Some TSRs do not work correctly when loaded to upper memory, resulting in crashs or erratic system behavior. If you notice any instability in your DOS system, try to move TSRs back into conventional memory to find the culprit. You may look for driver alternatives or in worst case need to change hardware. Mainboard BIOS, graphic card BIOS and also additional cards as mass storage controllers or network cards use the memory area between 640K and 1 MB sharing space with UMBs. Especially newer graphic cards and mainboards are not designed with DOS in mind and are delivered with large BIOSes (> 32 kB). This can reduce available UMBs considerably. A classic DOS system has about 128 kB to 192 kB free UMBs for TSRs. Using EMS emulation requires a 64 kB area between 640K and 1 MB where the page frame of a virtual EMS card is mapped. If not required EMS emulation can be disabled freeing 64 kB of UMBs.
Another way is to use a utility that changes the start up files such that the amount of free conventional memory is maximal (e.g. MS-DOS Memmaker, QEMM Optimize). Typically these programs run several analysis reboots to gather information about the drivers and memory layout. In the end a optimized configuration is presented. The analysis depth and the final quality of the presented new boot configuration of the programs varies. (todo: expand)
Software
HIMEM.SYS: enables XMS (except UMBs) with 286 up to 15 Mb, with 386/486 up to 1023 Mb
Versions:
MS-DOS 5.0 : 2.77, max. 15 Mb XMS
MS-DOS 6.0 : 3.07, recognizes max. 4 Gb, 1023 MB XMS useable.
MS-DOS 6.2 6.21 6.22 : 3.10, recognizes max. 4 Gb, 1023 MB XMS useable.
EMM386.EXE: introduced with MS-DOS 5.0, uses XMS to create UMBs in UMA, uses XMS to create EMS
MEMMAKER.EXE: introduced with MS-DOS 6.0, utility to optimize free conventional memory
QEMM386: a complete package of memory management driver and utilities, last version 9.0 / QEMM97
QEMM386.SYS memory management driver implementing XMS, EMS, UMA, VCPI and DPMI, enhanced features include support for up to 256 MB RAM, QuickBoot, Stealth, dynamically load drivers from command line and specific support for popular drivers as DoubleSpace, Stacker. Additional utilities include
Optimize a powerful tool to optimize free conventional memory
DOS-UP for relocating DOS to high memory
Manifest a system information tool with focus on memory layout
QDPMI a DPMI 0.9 server
MagnaRAM a memory compression utility replacing a portion of Windows 3.1 virtual memory subsystem.
UMBPCI Upper memory driver, partial replacement for EMM386.EXE. Uses Shadow-RAM instead of extended memory to create UMBs. Functionality is chipset dependent, see UMBPCI homepage for a list of supported chipsets. Also be aware of issues with ISA-DMA using the Shadow-RAM method that could lead to data corruption if not setup properly. This driver leaves the CPU in Real mode.
