High memory area

From Wikipedia the free encyclopedia

Not to be confused with High memory.

The high memory area is highlighted.

In DOS memory management, the high memory area (HMA) is the RAM area consisting of the first 65520 bytes above the one megabyte in an IBM AT or compatible computer.

In real mode, the segmentation architecture of the Intel 8086 and subsequent processors identifies memory locations with a 16-bit segment and a 16-bit offset, which is resolved into a physical address via (segment) × 16 + (offset). Although intended to address only 1 Megabyte (MB) (220 bytes) of memory, segment:offset addresses at FFFF:0010 and beyond reference memory beyond 1 MB (FFFF0 + 0010 = 100000). So, on an 80286 and subsequent processors, this mode can actually address the first 65520 bytes of extended memory as part of the 64 KB range starting 16 bytes before the 1 MB mark—FFFF:0000 (0xFFFF0) to FFFF:FFFF (0x10FFEF). The Intel 8086 and 8088 processors, with only 1 MB of memory and only 20 address lines, wrapped around at the 20th bit, so that address FFFF:0010 was equivalent to 0000:0000.[1]

To allow running existing DOS programs which relied on this feature to access low memory on their newer IBM PC AT computers, IBM added special circuitry on the motherboard to simulate the wrapping around. This circuit was a simple logic gate which could disconnect the microprocessor's 21st addressing line, A20, from the rest of the motherboard. This gate could be controlled, initially through the keyboard controller, to allow running programs which wanted to access the entire RAM.[1]

So-called A20 handlers could control the addressing mode dynamically,[1] thereby allowing programs to load themselves into the 1024–1088 KB region and run in real mode.[1]

Code suitable to be executed in the HMA must either be coded to be position-independent (using only relative references),[2][1] be compiled to work at the specific addresses in the HMA (typically allowing only one or at most two pieces of code to share the HMA), or it must be designed to be paragraph boundary or even offset relocatable (with all addresses being fixed up during load).[2][1]

Before code (or data) in the HMA can be addressed by the CPU, the corresponding driver must ensure that the HMA is mapped in. This requires that any such requests are tunneled through a stub remaining in memory outside the HMA, which would invoke the A20 handler in order to (temporarily) enable the A20 gate.[2][1] If the driver does not exhibit any public data structures and only uses interrupts or calls already controlled by the underlying operating system, it might be possible to register the driver with the system in a way so that the system will take care of A20 itself thereby eliminating the need for a separate stub.[1][nb 1]

The first user of the HMA among Microsoft products was Windows/286 2.1 in 1988, which introduced the HIMEM.SYS device driver. Starting in 1990 with Digital Research's DR DOS 5.0[3] (via HIDOS.SYS /BDOS=FFFF[4] and CONFIG.SYS HIDOS=ON) and since 1991 with MS-DOS 5.0[3] (via DOS=HIGH), parts of the operating system's BIOS and kernel could be loaded into the HMA as well,[3][5] freeing up to 46 KB of conventional memory.[1] Other components, such as device drivers and terminate-and-stay-resident programs (TSRs), could at least be loaded into the upper memory area (UMA), but not into the HMA. Under DOS 5.0 and higher, with DOS=HIGH, the system additionally attempted to move the disk buffers into the HMA.[5] Under DR DOS 6.0 (1991) and higher, the disk buffers (via HIBUFFERS, and later also BUFFERSHIGH), parts of the command processor COMMAND.COM as well as several special self-relocating drivers like KEYB, NLSFUNC and SHARE could load into the HMA as well (using their /MH option), thereby freeing up even more conventional memory and upper memory for conventional DOS software to work with.[1] TASKMAX seems to have relocated parts of itself into the HMA as well.[6][7] Novell's NLCACHE from NetWare Lite and early versions of NWCACHE from Personal NetWare and Novell DOS 7 could utilize the HMA as well.[8][9][7] Under MS-DOS/PC DOS, a ca. 2 KB shared portion of COMMAND.COM can be relocated into the HMA,[10] as well as DISPLAY.SYS bitmaps for prepared codepages.[10][11] Under MS-DOS 6.2 (1993) and higher, a ca. 5 KB portion of DBLSPACE.BIN/DRVSPACE.BIN can coexist with DOS in the HMA (unless DBLSPACE/DRVSPACE /NOHMA is invoked).[5][12] Under PC DOS 7.0 (1995) and 2000, DOSKEY loads into the HMA (if available),[13] and SHARE can be loaded into the HMA as well (unless its /NOHMA option is given).[13] Under MS-DOS 7.0 (1995) to 8.0 (2000), parts of the HMA are also used as a scratchpad to hold a growing data structure recording various properties of the loaded real-mode drivers.[7][14][15]

See also

[edit]

Notes

[edit]
  1. ^ A stub is not necessary for the resident system extensions SHARE and NLSFUNC under DR DOS 6.0 and higher, because they only hook the multiplex interrupt INT 2Fh and can therefore utilize a backdoor interface to hook into the interrupt chain in kernel space so that the kernel's gate A20 handler will provide the functionality of the stub.

References

[edit]
  1. ^ a b c d e f g h i j Paul, Matthias R. (2002-02-02). "Treiber dynamisch nachladen (Intra-Segment-Offset-Relokation zum Laden von TSRs in die HMA)" [Loading drivers dynamically (Intra-segment offset relocation to load TSRs into the HMA)] (in German). Newsgroup: de.comp.os.msdos. Archived from the original on 2017-09-09. Retrieved 2017-07-02. (NB. Gives a comprehensive overview on the history and "nature" of the HMA and the non-obvious design constraints to be observed when developing resident system extensions to be loaded into the HMA. It also describes how to address these issues using stubs, backdoors, and intra-segment offset relocation, a method used by DR-DOS drivers capable of relocating into the HMA and similar to a (more sophisticated) method used as the basis for the dynamic dead code elimination in the author's FreeKEYB driver.)
  2. ^ a b c Ingenoso, Tony (1998-12-20). "Chapter 13 - The A20 gate and the HMA". Making Code Work Better - How to minimize the size of 80x86 code and sometimes make it faster (e-book). Archived from the original on 2019-11-18. Retrieved 2019-11-18.
  3. ^ a b c Dryfoos, Mike, ed. (1991-09-18) [1991-07-19]. "MS-DOS 5.0 Development Post-Mortem Report" (PDF) (mail as court document). Microsoft. p. 10. MS-PCA1179169 (MS-PCA1179159-MS-PCA1179191). MS7020988 (MS7020978-MS7021010). Depo. Ex. 1109. Comes v Microsoft Plaintiff's Exhibit 3473. CA.No.2:96CV645B Plaintiff's Exhibit 477. Archived (PDF) from the original on 2019-04-02. Retrieved 2019-07-22. […] One of the most important stimulanta for adding features was competitive pressure from DRDOS 5.0, which we first learnt of in the spring of 1990. The DRDOS feature set led us to add UMB support, task swapping, and Undelete. […] Considerable amounts of the team's management attention was diverted to new features such as file transfer software, undelete and network installation […] Eventually this situation reached a crisis point at the end of July 1990, and, led by BradS, the team's management spent an arduous series of meetings nailing down a schedule and process for closing the project down […] (1+32 pages)
  4. ^ Banta, K.; Partridge, D. (1994-08-18). "Third Party Memory Managers". DR DOS 6.0 (Technical information document). Revision A. Novell. TID800074 (replaces FYI-M-1303). Archived from the original on 2021-12-15. Retrieved 2021-12-15.
  5. ^ a b c Schulman, Andrew; Brown, Ralf D.; Maxey, David; Michels, Raymond J.; Kyle, Jim (1994) [November 1993]. Williams, Andrew (ed.). Undocumented DOS: A programmer's guide to reserved MS-DOS functions and data structures - expanded to include MS-DOS 6, Novell DOS and Windows 3.1. The Andrew Schulman Programming Series (1st printing, 2nd ed.). Reading, Massachusetts, USA: Addison Wesley Publishing Company. pp. 42, 349–350, 437–438. ISBN 0-201-63287-X. ISBN 978-0-201-63287-3. (xviii+856+vi pages, 3.5"-floppy [1]) Errata: [2][3]
  6. ^ "Format of HMA Memory Block (DR DOS 6.0 kernel loaded in HMA)". RBIL. 2000. Archived from the original on 2020-02-18. Retrieved 2020-02-18.
  7. ^ a b c Paul, Matthias R. (2002-04-10). "[fd-dev] HMA access from TSR". freedos-dev. Archived from the original on 2017-09-09. Retrieved 2017-09-09. […] MS-DOS 7.0+ adds INT 21h/AX=4A03h and INT 21h/AX=4A04h. RBIL61 INT 21h/AH=52h has some info on the MS-DOS 7.0+ HMA MCB chain […] HMA relocation for TSRs makes much sense for DR-DOS: Although you can load large parts of the BIOS and BDOS, the resident part of the shell, the BUFFERS, and DR-DOS TSRs like SHARE, KEYB, and NLSFUNC (and in some issues parts of TASKMGR and NWCACHE) into the HMA, there is usually still free space available, typically around 10 Kb (up to ca. 20 Kb when you use a 3rd party shell). It also makes sense for MS-DOS 5.0 - 6.22 and PC DOS up to 2000, which typically leave 4 - 7 Kb of the HMA memory unused (SHARE, KEYB, and NLSFUNC cannot load into the HMA, but DBLSPACE and HIMEM can to some extent). Available HMA space can be rather tight with MS-DOS 7.0+, since this issue introduced a new and for the most part undocumented RMD data structure usually located in the HMA. The kernel collects and records configuration and Real Mode Driver data during boot (type of driver, interrupts hooked by driver, CONFIG.SYS line of invocation, etc.) and stores this information in an […] complicated […] and […] growing data structure. Presumably this info is meant to be used by the Windows core to get a better picture of the loaded Real Mode drivers instead of treating DOS as a monolithic block, or even […] attempt to unhook or unload some of them, however, it is only used to a very limited extent (for example you can see some of the info reflected in the log files created on Windows 9x startup, and some parts of the Windows configuration manager also make use of it), leaving room for speculation much beyond the technical side - in particular because nothing of the interesting stuff is documented… […]
  8. ^ Paul, Matthias R. (1997-07-30) [1994-05-01]. "II.4. Undokumentierte Eigenschaften externer Kommandos". NWDOS-TIPs — Tips & Tricks rund um Novell DOS 7, mit Blick auf undokumentierte Details, Bugs und Workarounds. MPDOSTIP (in German) (3 ed.). Archived from the original on 2016-11-05. Retrieved 2012-01-11. (NB. NWDOSTIP.TXT is a comprehensive work on Novell DOS 7 and OpenDOS 7.01, including the description of many undocumented features and internals. It is part of the author's yet larger MPDOSTIP.ZIP collection maintained up to 2001 and distributed on many sites at the time. The provided link points to a HTML-converted older version of the file.) [4]
  9. ^ Paul, Matthias R. (2001-04-09). "II.4. Undokumentierte Eigenschaften externer Kommandos". NWDOS-TIPs — Tips & Tricks rund um Novell DOS 7, mit Blick auf undokumentierte Details, Bugs und Workarounds. MPDOSTIP (in German) (3 ed.).
  10. ^ a b Chappell, Geoff (January 1994). Schulman, Andrew; Pedersen, Amorette (eds.). DOS Internals. The Andrew Schulman Programming Series (1st printing, 1st ed.). Addison Wesley Publishing Company. pp. 4, 21, 100–106, 127–129. ISBN 978-0-201-60835-9. ISBN 0-201-60835-9. (xxvi+738+iv pages, 3.5"-floppy [5][6]) Errata: [7][8][9]
  11. ^ Paul, Matthias R. (2002-12-04). "[fd-dev] DISPLAY CON?". freedos-dev. Archived from the original on 2021-12-08. […] some issues of DISPLAY.SYS (of PC DOS 7/2000, for example) store the currently unused fonts in XMS memory. Some earlier issues of MS-DOS/PC DOS DISPLAY.SYS seem to have had a facility to store them in the HMA […]
  12. ^ Cooper, Jim (2002). Using MS-DOS 6.22 (special 3rd ed.). Que Publishing. p. 669. ISBN 0-78972573-8. ISBN 978-0-78972573-8. Archived from the original on 2020-02-18. Retrieved 2020-02-18.
  13. ^ a b Brooks, Vernon C. (2014). "This is a detailed list of the changes I made in PC DOS 7.0". PC DOS Retro. Archived from the original on 2020-02-18. Retrieved 2020-02-18. […] DOSKEY.COM […] Move code to HMA if available. […] SHARE.EXE […] Move code to HMA if available and added /NOHMA option force loading low. […]
  14. ^ Sweger, Kristofer (2007) [2002-07-15]. "ANSIPLUS and Windows". Archived from the original on 2021-11-28. Retrieved 2021-11-28. […] ANSIPLUS's code cannot be loaded to the HMA under MS-DOS 7 (Windows 9x only) because there apparently is not enough unused HMA memory available. […]
  15. ^ Paul, Matthias R. (2002-08-13). "Suche freien Speicherbereich unterhalb von 1 MB, der nicht von OS überschrieben wird" (in German). Newsgroup: de.comp.lang.assembler.x86. Archived from the original on 2017-09-04. Retrieved 2017-09-03.
  16. ^ Paul, Matthias R. (1997-10-02) [1997-09-29]. "Caldera OpenDOS 7.01/7.02 Update Alpha 3 IBMBIO.COM - README.TXT and BOOT.TXT - A short description of how OpenDOS is booted". Archived from the original on 2003-10-04. Retrieved 2009-03-29. [10]

Further reading

[edit]
  • Necasek, Michal (2011-09-13). "Who needs the address wraparound, anyway?". OS/2 Museum. Archived from the original on 2020-02-19. Retrieved 2020-02-19. […] 86-DOS, and hence PC DOS/MS-DOS, used a clever trick. The byte at offset 5 of the PSP contained a far call opcode (9Ah); the word at offset 6 of the PSP contained the appropriate value to indicate program segment size, and also the offset part of the far call. The word at offset 8, which served as the segment part of the far call, was crafted such that when combined with the offset, it would wrap around (a well understood feature of the 8086 CPU) and point to address 0:C0h, which contains interrupt vector 30h. […] the CALL 5 interface works even in DOS emulation under Windows NT and OS/2, and those systems most certainly cannot run with the A20 line disabled. How does that work then? […] Rather than chopping off address bits, the system mirrors the five bytes at 0:C0h at 1000C0h. The same technique had been in fact used in DOS 5 and above running with DOS=HIGH. In that case, DOS makes sure that linear address 1000C0h contains the appropriate far call. […]
  • Kozierok, Charles M. (2001-04-17) [1997]. "High Memory Area (HMA)". The PC Guide. 2.2.0. Archived from the original on 2006-10-16. Retrieved 2006-10-15.
  • Paul, Matthias R. (2002-04-11). "Re: [fd-dev] ANNOUNCE: CuteMouse 2.0 alpha 1". freedos-dev. Archived from the original on 2020-02-21. Retrieved 2020-02-21. […] in case of such mangled pointers […] many years ago Axel and I were thinking about a way how to use *one* entry point into a driver for multiple interrupt vectors (as this would save us a lot of space for the multiple entry points and the more or less identical startup/exit framing code in all of them), and then switch to the different interrupt handlers internally. For example: 1234h:0000h […] 1233h:0010h […] 1232h:0020h […] 1231h:0030h […] 1230h:0040h […] all point to exactly the same entry point. If you hook INT 21h onto 1234h:0000h and INT 2Fh onto 1233h:0010h, and so on, they would all go through the same "loophole", but you would still be able to distinguish between them and branch into the different handlers internally. Think of a "compressed" entry point into a A20 stub for HMA loading. This works as long as no program starts doing segment:offset magics. […] Contrast this with the opposite approach to have multiple entry points (maybe even supporting IBM's Interrupt Sharing Protocol), which consumes much more memory if you hook many interrupts. […] We came to the result that this would most probably not be save in practise because you never know if other drivers normalize or denormalize pointers, for what reasons ever. […]
Retrieved from "https://en.wikipedia.org/w/index.php?title=High_memory_area&oldid=1226612829"