Difference between revisions of "DOS memory management"

Revision as of 11:03, 11 April 2013

With modern operating systems, all memory management you need to do is installing enough RAM. And that's it. The OS determines how to use it, and you usually don't need to worry about it at all. In DOS hardware and memory access is closely related to the standards defined by the x86 architecture. In DOS memory is split into different regions and there are several standards as EMS and XMS for accessing memory. To use the available RAM effectively in DOS you have to know how the x86 architecture splits up RAM, what these different regions are for. It is also helpful to know the most common memory management drivers. Optimizing your system to attain enough free memory for your programs is called memory management.

This page will try to explain, in simple language, all the technical terms you will encounter in DOS memory management, and provide you with practical information to get your favorite games and programs running. If you really want to understand how this all works, see the Wikipedia article on DOS memory management, or use your web search engine of choice.

Types of memory

Conventional memory

Conventional memory or base memory is the memory range between 0 kb and 640 kb. Programs are loaded to this memory range. The available free memory can be lower as some drivers have to leave a part of their code in memory to handle e.g. hardware. This is called Terminate and Stay Resident (TSR). By default such drivers are placed in conventional memory. The amount of free conventional memory is reported by CHKDSK or MEM.

Upper memory

The upper memory area (UMA) is memory in the range between 640 kb and 1 Mb. By default there is no RAM in this range as it is reserved for use with hardware that is able to map own memory to this range. Usually present in this region is a part of the graphics cards RAM and the BIOS ROMs of the graphics card and mainboard. Additional hardware like mass storage controllers, network adapters... can use parts of the UMA for own BIOS ROMs or buffer RAM. The Extended Memory Specification (XMS) allows to map blocks of mainboard RAM in the remaining UMA memory range. These blocks are called Upper Memory Blocks (UMB) and are treated from DOS as High Memory. The default memory management driver that enables this mapping through XMS memory is EMM386.EXE. It has to be loaded in config.sys as second driver after HIMEM.SYS. The driver tries to detect common unused blocks and maps XMS memory there creating Upper Memory Blocks. By specifying DOS=UMB in config.sys MS-DOS allocates all UMBs through XMS and takes over memory management of them. this allows TSR-programs to be loaded to UMBs (in High Memory) in config.sys by using DEVICEHIGH=driver.sys or in autoexec.bat with the loadhigh statement, like LH driver.com. If no contiguous free Upper Memory Block is available the driver will be loaded to Conventional Memory. Since UMA memory is managed in blocks the amount of free Upper Memory is usually larger than the largest contiguous free block. The external program MEM with parameter /C reports additionally the UMA RAM status.

High memory area

The High Memory Area (HMA) are the 64 kb directly above 1 Mb and are part of XMS. To use HMA it is required to load HIMEM.SYS in config.sys. By specifying the line DOS=HIGH in config.sys DOS is asked to load parts to HMA. Directly after an XMS driver is loaded in config.sys with DEVICE= and the HMA is still unoccupied DOS will move parts of code there. If DOS=HIGH was specified but loading to HMA fails HMA not available/loading DOS low is reported. The external program MEM reports if DOS is using the HMA.

Expanded memory

The Expanded Memory Specification (EMS) is a standard developed by Lotus, Intel and Microsoft. Expanded Memory can be either memory on an memory expansion card or a part of the main memory. The specification describes that this memory can be used by mapping a 64 kB large part to the Upper Memory Area between 640 kB and 1 Mb. The 64 kB region within the Upper Memory Area is referred to as EMS Page Frame. The latest EMS 4.0 allowed to use up to 32 Mb as expanded memory. The most commonly used memory management driver implementing EMS is EMM386.EXE that has to be loaded in config.sys as second driver after HIMEM.SYS. EMM386 emulates expanded memory by using main memory beyond the High Memory Area. To achieve this it has to switch the CPU to Protected Mode. The amount of free main memory beyond 1 MB can be shared between EMS and XMS by adding the parameter AUTO to EMM386.EXE. In the case no EMS is required this feature can be deactivated by using the parameter NOEMS. This frees 64 kB in the Upper Memory Area for loading TSR-programs.

Versions:

EMS 3.0: max. 4 Mb

EMS 3.2: max. 8 Mb

EMS 4.0: max. 32 Mb

Extended memory

The Extended Memory Specification (XMS) describes an application interface that allows to copy memory between conventional and extended memory. Extended Memory is all main memory beyond 1 Mb. The most common memory management driver for this functionality is HIMEM.SYS that has to be loaded as first driver in config.sys. Accessing memory above the High Memory Area requires switching the CPU to Protected Mode, thus HIMEM.SYS requires at least a 286 CPU.

Versions:

XMS 2.0: max. 64 Mb

XMS 3.0: max. 4 Gb

Additional Remarks

While UMBs are a optional feature of XMS HIMEM.SYS does not support UMB. On bare MS-DOS this requires to load EMM386.EXE.

The DOS=HIGH and DOS=UMB statements in config.sys can be combined by writing DOS=HIGH,UMB. Specifying DOS=LOW can be used to prevent DOS from using the HMA.

Shadow RAM is RAM in the same range as BIOS ROMs. It can usually enabled by BIOS or as parameter of a memory manager. Since ROM access is rather slow it is usually used to copy the BIOS ROM to the underlying RAM at the same location and then switching off the ROM. This accelerates all BIOS calls.

Architectural reasons force the memory for DMA transfers of ISA cards to be reserved below 1 Mb. This is usually of 16 kB size and reserved in UMA.

Memory management

Getting more conventional memory

A challenging aspect of DOS memory management can be to free enough conventional memory. Often games require a high amount of free conventional memory and sometimes also free EMS or XMS. To achieve this you have to use memory managers that allow e.g. enabling of UMBs and installation of APIs for XMS and EMS access. The availability of memory types depends on CPU and chipset.

On 286 CPUs special chipset support is required to enable UMBs. Rather well supported is the NEAT chipset. The HIMEM.SYS driver takes control of the A20-gate, makes HMA available and installs a XMS API. It is also possible to emulate a EMS page frame with the EMM286 driver. Some chipsets have jumpers to set XMS and EMS assignment of the main memory.

386 and later CPUs feature an protected mode extension called Virtual 8086 mode that allows mimicking a 8086 memory layout for applications. With MS-DOS 5 a second memory manager was included called EMM386.EXE that uses the advanced features of 386 CPUs. It is loaded after HIMEM.SYS and enables UMBs and EMS emulation. This allows to load TSRs to UMBs freeing conventional RAM. The corresponding load statement in config.sys is DEVICEHIGH= and in autoexec.bat LOADHIGH which can be shortened to LH. EMM386 switches the CPU to Virtual 8086 mode.

(todo UMBPCI)

Generally memory managers must be loaded first. As rule of thumb TSRs taking more memory should be loaded before small TSRs to prevent memory fragmentation. Some drivers require a specific load order e.g. SMARTDRV.EXE after MSCDEX.EXE if the CD-ROM should be read cached. Some TSRs do not work correctly when loaded to upper memory, resulting in crashs or erratic system behavior. If you notice any instability in your DOS system, try to move TSRs back into conventional memory to find the culprit. You may look for driver alternatives or in worst case need to change hardware. Mainboard BIOS, graphic card BIOS and also additional cards as mass storage controllers or network cards use the memory area between 640K and 1 MB sharing space with UMBs. Especially newer graphic cards and mainboards are not designed with DOS in mind and are delivered with large BIOSes (> 32 kB). This can reduce available UMBs considerably. A classic DOS system has about 128 kB to 192 kB free UMBs for TSRs. Using EMS emulation requires a 64 kB area between 640K and 1 MB where the page frame of a virtual EMS card is mapped. If not required EMS emulation can be disabled freeing 64 kB of UMBs.

Another way is to use a utility that changes the start up files such that the amount of free conventional memory is maximal (e.g. MS-DOS Memmaker, QEMM Optimize). Typically these programs run several analysis reboots to gather information about the drivers and memory layout. In the end a optimized configuration is presented. The analysis depth and the final quality of the presented new boot configuration of the programs varies. (todo: expand)

Software

HIMEM.SYS: enables XMS (except UMBs) with 286 up to 15 Mb, with 386/486 up to 1023 Mb

Versions:

MS-DOS 5.0 : 2.77, max. 15 Mb XMS

MS-DOS 6.0 : 3.07, recognizes max. 4 Gb, 1023 MB XMS useable.

MS-DOS 6.2 6.21 6.22 : 3.10, recognizes max. 4 Gb, 1023 MB XMS useable.

EMM386.EXE: introduced with MS-DOS 5.0, uses XMS to create UMBs in UMA, uses XMS to create EMS

MEMMAKER.EXE: introduced with MS-DOS 6.0, utility to optimize free conventional memory

QEMM386: a complete package of memory management driver and utilities, last version 9.0 / QEMM97

QEMM386.SYS memory management driver implementing XMS, EMS, UMA, VCPI and DPMI, enhanced features include support for up to 256 MB RAM, QuickBoot, Stealth, dynamically load drivers from command line and specific support for popular drivers as DoubleSpace, Stacker. Additional utilities include

Optimize a powerful tool to optimize free conventional memory

DOS-UP for relocating DOS to high memory

Manifest a system information tool with focus on memory layout

QDPMI a DPMI 0.9 server

MagnaRAM a memory compression utility replacing a portion of Windows 3.1 virtual memory subsystem.

Overview of Memory-Management Functionality in MS-DOS

Overview about features of different memory managers