================[ Recap of x86 hardware page tables ]================

This blog post by Gustavo Duarte has a brief summary of how Linux
uses x86 hardware page tables:
http://duartes.org/gustavo/blog/post/how-the-kernel-manages-your-memory/

Look at the beautiful picture of a PTE while you read my 
transcript below and see the PTE values parsed.

Previous useful background links from G. Duarte's blog:
http://duartes.org/gustavo/blog/post/memory-translation-and-segmentation/
http://duartes.org/gustavo/blog/post/cpu-rings-privilege-and-protection/

================[ Walking x86 page tables in MDB ]================

The book describes the x86_64 pages tables in Ch. 12.3. We will walk
the page tables of a 32-bit address space (of the mmap.c process);
do the same for a 64 bit process of your choice.

root@openindiana:/ # mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs sata sd ip hook neti sockfs arp usba stmf stmf_sbd lofs random idm crypto sppp ipc ptm ]
> ::pgrep mmap
S    PID   PPID   PGID    SID    UID      FLAGS             ADDR NAME
R   1686   1647   1686   1647    101 0x4a004000 ffffff0159665050 mmap

Let's navigate to the address space struct:

> ffffff0159665050::print -t proc_t
proc_t {
    struct vnode *p_exec = 0xffffff0159716c00
    struct as *p_as = 0xffffff0158223110
<skip>

> 0xffffff0158223110::print -t 'struct as'
struct as {
    kmutex_t a_contents = {
        void *[1] _opaque = [ 0 ]
    }
    uchar_t a_flags = 0
    uchar_t a_vbits = 0
    kcondvar_t a_cv = {
        ushort_t _opaque = 0
    }
    struct hat *a_hat = 0xffffff0157c3d230
<skip>

The HAT struct abstracts the actual page tables of a process, one per
address space. A lower-level abstraction, struct htable, abstracts
and indexes the individual page tables (of which there are 4 levels
in x86_64; there is one htable per level).

All these structures are linked in hash tables, so that you could
look up in which processes and actual page tables an address is
mapped, both forward and in reverse (read about this in the book,
paying extra attention to hment_t structures, and find multiply
mapped pages across processes. You can use the page_t arrays of
memsegs for this, too.)

Let's examine the HAT for this process:

> 0xffffff0157c3d230::print -t 'struct hat'
struct hat {
    kmutex_t hat_mutex = {
        void *[1] _opaque = [ 0 ]
    }
    struct as *hat_as = 0xffffff0158223110      <--- back reference to owning AS
    uint_t hat_stats = 0
    pgcnt_t [3] hat_pages_mapped = [ 0x109, 0, 0xbaddcafebaddcafe ]
    pgcnt_t hat_ism_pgcnt = 0
    cpuset_t hat_cpus = {
        ulong_t [4] cpub = [ 0, 0, 0, 0 ]
    }
    uint16_t hat_flags = 0x2
    htable_t *hat_htable = 0xffffff015941e468    <--- next level of page table abstraction
    struct hat *hat_next = 0xffffff0149472638    <--- linkage of HAT structs
    struct hat *hat_prev = 0xffffff0157c3d2e0
    uint_t hat_num_hash = 0x200
    htable_t **hat_ht_hash = 0xffffff0159664000  <--- more linkage
    htable_t *hat_ht_cached = 0
    x86pte_t [4] hat_vlp_ptes = [ 0x73f02027, 0, 0, 0x4f904027 ]
}

The last entry, hat_vlp_ptes, is especially important for 32-bit address spaces.
To save page table space, these entries are the _only_ entries for the top-level
(level 3) of the memory translation hierarchy, and are kept in the per-CPU area.
Thus the CR3 register of each CPU running threads in the process points to
the table of these four entries.

Let's examine these:

> 0x73f02027::pte
pte=73f02027: pfn=0x73f02 user write 

> 0x4f904027::pte
pte=4f904027: pfn=0x4f904 user write 

See http://wiki.osdev.org/Paging for layouts of PDEs and PTEs. The last three nybbles
are flags (slightly different in meaning between the PTEs and PDEs). The ::pte command
interprets some of these flags; for example, the lowers nybble of 7 means that
U, W, and P flags are set => the page is present, writable, and "user" (not supervisor;
expect this bit to be set only for the SMEP/SMAP feature of the Intel MMU,
explained in https://forums.grsecurity.net/viewtopic.php?f=7&t=3046 )

The PFNs in these entries are the physical pages in which the next level tables reside.
The ::ptable command interprets these, skipping null entries (which tend to dominate in
real page tables).

> 0x4f904::ptable
htable=ffffff015941e6a8
[503] va=fee00000 pte=69785027: pfn=0x69785 ref user write 

This level-2 table has only one entry (in its entire 4K page frame). To wit:

> 0x4f904000,200\K
0x4f904000:     0               0               0               0               
                0               0               0               0               
<skip 120 lines like that> 
                0               0               0               0               
                0               0               0               0               
                0               0               0               0               
                0               0               0               69785027        
                0               0               0               0               
                0               0               0               0               

Note that the pointer to the htable_t abstraction is also helpfully provided by ::ptable:

> 0x4f904::ptable
htable=ffffff015941e6a8
[503] va=fee00000 pte=69785027: pfn=0x69785 ref user write 

The HTABLE structures correspond to individual page directory (PD) and page table
(PT) pages, one per contiguous table or directory. Note that---unlike the underlying
PDs and PTs---these HTABLE structs are linked in multiple ways, to provide for
looking up the AS structure and the AVL tree segment the fauling address belongs to. 

Suggestion:
Walk these hash tables and chains to get from an address to the physical page and back.

> ffffff015941e6a8::print -t htable_t
htable_t {
    struct htable *ht_next = 0
    struct hat *ht_hat = 0xffffff0157c3d230    <----- owning HAT struct
    uintptr_t ht_vaddr = 0xc0000000            <----- address that would be mapped by the first entry of this page table
    int8_t ht_level = '\001'                   <----- level of this page table, counting backward (one level of translation remains)
    uint8_t ht_flags = 0
    int16_t ht_busy = 0x1
    int16_t ht_valid_cnt = 0x1
    uint32_t ht_lock_cnt = 0
    pfn_t ht_pfn = 0x4f904                     <------ PFN of the phys page where the actual PT/PD is (what matters to hardware)
    struct htable *ht_prev = 0
    struct htable *ht_parent = 0xffffff015941e468   <---- link to parent table
    struct htable *ht_shares = 0
}

Looking at the parent htable_t:

> 0xffffff015941e468::print -t htable_t
htable_t {
    struct htable *ht_next = 0
    struct hat *ht_hat = 0xffffff0157c3d230   <------ same owning HAT (there's only one per process)
    uintptr_t ht_vaddr = 0
    int8_t ht_level = '\002'                  <------ two levels of translations remain
    uint8_t ht_flags = 0x1
    int16_t ht_busy = 0x3
    int16_t ht_valid_cnt = 0x2
    uint32_t ht_lock_cnt = 0
    pfn_t ht_pfn = 0xffffffffffffffff   <---- this top-level table is located in the per-CPU area (see hat_vlp_ptes above)
    struct htable *ht_prev = 0
    struct htable *ht_parent = 0        <---- no parent abstraction, no parent page table
    struct htable *ht_shares = 0
}

Same for the other level-2 page table:

> 0x73f02::ptable
htable=ffffff015941e4b0
[ 64] va=8000000 pte=5dd03027: pfn=0x5dd03 ref user write 

Let's go to the next level:

> 0x5dd03::ptable
htable=ffffff015941e3d8
[ 70] va=8046000 pte=72ad7067: pfn=0x72ad7 user write 
[ 71] va=8047000 pte=4c458067: pfn=0x4c458 user write 
[ 80] va=8050000 pte=68561025: pfn=0x68561 user 
[ 96] va=8060000 pte=69f71067: pfn=0x69f71 user write 
[ 97] va=8061000 pte=6c272067: pfn=0x6c272 user write 

We can look at this page table in raw physical memory as well (isn't
Illumos' MDB truly wonderful this way?):

> 0x5dd03000,200\K
0x5dd03000:     0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                0               0               72ad7067        4c458067
                0               0               0               0
                0               0               0               0
                68561025        0               0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
                69f71067        6c272067        0               0
                0               0               0               0
                0               0               0               0
                0               0               0               0
<skipping 100 more lines of zeros>

Looking at the first non-zero entry:

> 72ad7067::pte
pte=72ad7067: pfn=0x72ad7 user write   <--- also Dirty, Accessed, and Present.

This is the last level of PTEs. The PFNs in it are those of the actual
physical pages holding the process' memory. I made an off-by-one
counting mistake in class, and tried to interpret one of these pages
(last-but-one above) as a page table. This is what I got:

> 0x69f71::ptable
Unknown pagetable - assuming level/addr 0[  0] va=0 pte=6010101464c457f: pfn=0x
10101464c4 noconsist global user write uncached
[  1] va=1000 pte=1: pfn=0x0 
[  2] va=2000 pte=100030002: pfn=0x100030 write  !VALID 
[  3] va=3000 pte=3408050b8c: pfn=0x3408050 nosync global user wrthru !VALID 
[  4] va=4000 pte=1c88: pfn=0x1 noconsist wrthru !VALID 
[  5] va=5000 pte=28000700200034: pfn=0x8000700200 user uncached !VALID 
[  6] va=6000 pte=600200021: pfn=0x600200 
[  7] va=7000 pte=805003400000034: pfn=0x5003400000 user uncached !VALID 
[  8] va=8000 pte=e000000000: pfn=0xe000000  !VALID 
[  9] va=9000 pte=5000000e0: pfn=0x500000 wrback !VALID 
[ 10] va=a000 pte=300000000: pfn=0x300000  !VALID 
[ 11] va=b000 pte=805011400000114: pfn=0x5011400000 global user uncached !VALID 
[ 12] va=c000 pte=1100000000: pfn=0x1100000  !VALID 
[ 13] va=d000 pte=400000011: pfn=0x400000 uncached
[ 14] va=e000 pte=6ffffffd00000000: pfn=0xffffd00000  !VALID 
[ 15] va=f000 pte=805012800000128: pfn=0x5012800000 global wrthru !VALID 
[ 16] va=10000 pte=1000000000: pfn=0x1000000  !VALID 
[ 17] va=11000 pte=400000010: pfn=0x400000 uncached !VALID 
[ 18] va=12000 pte=100000000: pfn=0x100000  !VALID 
[ 19] va=13000 pte=805000000000000: pfn=0x5000000000  !VALID 
[ 20] va=14000 pte=ece00000000: pfn=0xece00000  !VALID 
[ 21] va=15000 pte=500000ece: pfn=0x500000 noconsist nosync user write wrthru

So, clearly wrong with weird entries and some flags that aren't normally set, of course.

OK, back to that actual last-level page table:

> 0x5dd03::ptable
htable=ffffff015941e3d8
[ 70] va=8046000 pte=72ad7067: pfn=0x72ad7 user write 
[ 71] va=8047000 pte=4c458067: pfn=0x4c458 user write 
[ 80] va=8050000 pte=68561025: pfn=0x68561 user 
[ 96] va=8060000 pte=69f71067: pfn=0x69f71 user write 
[ 97] va=8061000 pte=6c272067: pfn=0x6c272 user write 

Note the "67" and "25" flags at the end of these PTEs. "67" means a page that
is Accessed and Dirty (6), as well as User, Writable, and Present. "25" means
a page that is Accessed (but not Dirty), as well as User and Present (but not
Writable). This explains why it's not Dirty---and also suggests it's code,
and indeed it is:

$ pmap `pgrep mmap`
1686:	./mmap
08046000       8K rwx--    [ stack ]
08050000       4K r-x--  /export/home/sergey/mmap/mmap     <--- Not writable, executable, likely code.
08060000       8K rwx--  /export/home/sergey/mmap/mmap
FEDF0000      12K rw---    [ anon ]
FEE00000      12K rw---    [ anon ]
FEE10000      24K rwx--    [ anon ]
FEE20000       4K rwx--    [ anon ]
FEE30000    1220K r-x--  /usr/lib/libc/libc_hwcap1.so.1
FEF71000      36K rwx--  /usr/lib/libc/libc_hwcap1.so.1
FEF7A000       8K rwx--  /usr/lib/libc/libc_hwcap1.so.1
FEF80000       4K rw---    [ anon ]
FEF90000       4K rw---    [ anon ]
FEFA0000       4K rwx--    [ anon ]
FEFB0000       4K rwx--    [ anon ]
FEFB7000     208K r-x--  /lib/ld.so.1
FEFFB000       8K rwx--  /lib/ld.so.1
FEFFD000       4K rwx--  /lib/ld.so.1
 total      1572K

Notice that the NX protection bits are not set on most of these
ranges, including the stack.  Let's see some segments where the NX bit
is actually set. Looking at the pmap above, we'd have to look in the
PDEs that map the FEF[89]* and FEE00000--FEDF0000 anon-mapped areas;
indeed, we see:

> 0x69785::ptable
htable=ffffff015941e420
[  0] va=fee00000 pte=800000005a991067: noexec pfn=0x5a991 user write   <---- these are the 3 pages we mapped and wrote with 0xdeadbeef
[  1] va=fee01000 pte=800000005c712067: noexec pfn=0x5c712 user write   <---- ditto
[  2] va=fee02000 pte=8000000075413067: noexec pfn=0x75413 user write   <---- ditto 
[ 16] va=fee10000 pte=61821067: pfn=0x61821 user write 
[ 17] va=fee11000 pte=6f422067: pfn=0x6f422 user write 
[ 18] va=fee12000 pte=5c423067: pfn=0x5c423 user write 
[ 32] va=fee20000 pte=5d031067: pfn=0x5d031 user write 
[ 48] va=fee30000 pte=1347025: pfn=0x1347 user        <---- these pages are mapped read-only (see pmaps above); observe "25" PTE flags
[ 49] va=fee31000 pte=76ac8025: pfn=0x76ac8 user      <---- and these below... (keep looking at pmaps)
[ 50] va=fee32000 pte=742c9025: pfn=0x742c9 user 
[ 51] va=fee33000 pte=75aca025: pfn=0x75aca user 
[ 52] va=fee34000 pte=73c4b025: pfn=0x73c4b user 
[ 53] va=fee35000 pte=7584c025: pfn=0x7584c user 
[ 54] va=fee36000 pte=7674d025: pfn=0x7674d user 
[ 55] va=fee37000 pte=75e4e025: pfn=0x75e4e user 
[ 63] va=fee3f000 pte=76ed6025: pfn=0x76ed6 user 
[ 64] va=fee40000 pte=73c57025: pfn=0x73c57 user 
[ 65] va=fee41000 pte=75e58025: pfn=0x75e58 user 
[ 66] va=fee42000 pte=75259025: pfn=0x75259 user 
[ 67] va=fee43000 pte=75e5a025: pfn=0x75e5a user 
[ 68] va=fee44000 pte=73cdb025: pfn=0x73cdb user 
[ 69] va=fee45000 pte=75adc025: pfn=0x75adc user 
[ 70] va=fee46000 pte=75edd025: pfn=0x75edd user 
[ 71] va=fee47000 pte=74ade025: pfn=0x74ade user 
[ 72] va=fee48000 pte=74a5f025: pfn=0x74a5f user 
[ 73] va=fee49000 pte=74ae0025: pfn=0x74ae0 user 
[ 74] va=fee4a000 pte=74a61025: pfn=0x74a61 user 
[ 75] va=fee4b000 pte=74ae2005: pfn=0x74ae2 user 
[ 76] va=fee4c000 pte=73c63025: pfn=0x73c63 user 
[ 77] va=fee4d000 pte=74ae4025: pfn=0x74ae4 user 
[ 78] va=fee4e000 pte=74ae5025: pfn=0x74ae5 user 
[ 79] va=fee4f000 pte=757e6025: pfn=0x757e6 user 
[ 80] va=fee50000 pte=76ee7025: pfn=0x76ee7 user 
[ 81] va=fee51000 pte=74a68025: pfn=0x74a68 user 
[ 82] va=fee52000 pte=757e9025: pfn=0x757e9 user 
[ 83] va=fee53000 pte=73c6a025: pfn=0x73c6a user 
[ 84] va=fee54000 pte=74a6b025: pfn=0x74a6b user 
[ 85] va=fee55000 pte=75aec005: pfn=0x75aec user      <----- ...up to here. This page has not been Accessed, through mapped 
[ 86] va=fee56000 pte=74a6d005: pfn=0x74a6d user      <-----    and there are more pages like that from here on, intermittently. 
[ 87] va=fee57000 pte=7506e005: pfn=0x7506e user 
[ 89] va=fee59000 pte=74a70025: pfn=0x74a70 user 
[ 90] va=fee5a000 pte=756f1025: pfn=0x756f1 user 
[ 91] va=fee5b000 pte=74e72025: pfn=0x74e72 user 
[ 92] va=fee5c000 pte=769f3005: pfn=0x769f3 user 
[ 93] va=fee5d000 pte=7af74005: pfn=0x7af74 user 
[ 94] va=fee5e000 pte=75b75025: pfn=0x75b75 user 
[ 95] va=fee5f000 pte=75ef6025: pfn=0x75ef6 user 
[ 96] va=fee60000 pte=77077005: pfn=0x77077 user 
[ 97] va=fee61000 pte=77364025: pfn=0x77364 user 
[ 98] va=fee62000 pte=768e5025: pfn=0x768e5 user 
[ 99] va=fee63000 pte=768e6005: pfn=0x768e6 user 
[100] va=fee64000 pte=742e7005: pfn=0x742e7 user 
[101] va=fee65000 pte=75ee8005: pfn=0x75ee8 user 
[102] va=fee66000 pte=76569005: pfn=0x76569 user 
[103] va=fee67000 pte=77cea005: pfn=0x77cea user 
[104] va=fee68000 pte=74deb005: pfn=0x74deb user 
[112] va=fee70000 pte=75b73025: pfn=0x75b73 user 
[113] va=fee71000 pte=75bf4025: pfn=0x75bf4 user 
[114] va=fee72000 pte=76175005: pfn=0x76175 user 
[115] va=fee73000 pte=78af6005: pfn=0x78af6 user 
[116] va=fee74000 pte=75e77005: pfn=0x75e77 user 
[117] va=fee75000 pte=76178005: pfn=0x76178 user 
[118] va=fee76000 pte=75579025: pfn=0x75579 user 
[119] va=fee77000 pte=755fa005: pfn=0x755fa user 
[120] va=fee78000 pte=739fb025: pfn=0x739fb user 
[121] va=fee79000 pte=75a7c025: pfn=0x75a7c user 
[122] va=fee7a000 pte=75bfd005: pfn=0x75bfd user 
[123] va=fee7b000 pte=7387e005: pfn=0x7387e user 
[124] va=fee7c000 pte=739ff005: pfn=0x739ff user 
[125] va=fee7d000 pte=73980005: pfn=0x73980 user 
[126] va=fee7e000 pte=73981005: pfn=0x73981 user 
[127] va=fee7f000 pte=73982005: pfn=0x73982 user 
[128] va=fee80000 pte=73983005: pfn=0x73983 user 
[132] va=fee84000 pte=73987025: pfn=0x73987 user 
[133] va=fee85000 pte=73988005: pfn=0x73988 user 
[134] va=fee86000 pte=73989005: pfn=0x73989 user 
[135] va=fee87000 pte=7398a005: pfn=0x7398a user 
[136] va=fee88000 pte=7398b005: pfn=0x7398b user 
[137] va=fee89000 pte=7398c005: pfn=0x7398c user 
[138] va=fee8a000 pte=7398d005: pfn=0x7398d user 
[139] va=fee8b000 pte=7398e005: pfn=0x7398e user 
[142] va=fee8e000 pte=73991025: pfn=0x73991 user 
[143] va=fee8f000 pte=73992005: pfn=0x73992 user 
[144] va=fee90000 pte=73993005: pfn=0x73993 user 
[145] va=fee91000 pte=73994005: pfn=0x73994 user 
[146] va=fee92000 pte=73995005: pfn=0x73995 user 
[147] va=fee93000 pte=73996005: pfn=0x73996 user 
[148] va=fee94000 pte=73997005: pfn=0x73997 user 
[149] va=fee95000 pte=73998005: pfn=0x73998 user 
[155] va=fee9b000 pte=74b9e025: pfn=0x74b9e user 
[156] va=fee9c000 pte=7549f025: pfn=0x7549f user 
[157] va=fee9d000 pte=26a0005: pfn=0x26a0 user 
[158] va=fee9e000 pte=75ea1005: pfn=0x75ea1 user 
[159] va=fee9f000 pte=77122005: pfn=0x77122 user 
[160] va=feea0000 pte=1ba3005: pfn=0x1ba3 user 
[161] va=feea1000 pte=76024005: pfn=0x76024 user 
[162] va=feea2000 pte=754a5005: pfn=0x754a5 user 
[175] va=feeaf000 pte=739b2025: pfn=0x739b2 user 
[176] va=feeb0000 pte=739b3005: pfn=0x739b3 user 
[177] va=feeb1000 pte=739b4005: pfn=0x739b4 user 
[178] va=feeb2000 pte=739b5005: pfn=0x739b5 user 
[179] va=feeb3000 pte=739b6005: pfn=0x739b6 user 
[180] va=feeb4000 pte=739b7005: pfn=0x739b7 user 
[181] va=feeb5000 pte=739b8005: pfn=0x739b8 user 
[182] va=feeb6000 pte=739b9005: pfn=0x739b9 user 
[191] va=feebf000 pte=739c2025: pfn=0x739c2 user 
[192] va=feec0000 pte=739c3025: pfn=0x739c3 user 
[193] va=feec1000 pte=739c4025: pfn=0x739c4 user 
[194] va=feec2000 pte=739c5025: pfn=0x739c5 user 
[195] va=feec3000 pte=739c6025: pfn=0x739c6 user 
[196] va=feec4000 pte=739c7025: pfn=0x739c7 user 
[197] va=feec5000 pte=739c8005: pfn=0x739c8 user 
[198] va=feec6000 pte=739c9005: pfn=0x739c9 user 
[199] va=feec7000 pte=739ca005: pfn=0x739ca user 
[200] va=feec8000 pte=739cb005: pfn=0x739cb user 
[201] va=feec9000 pte=739cc005: pfn=0x739cc user 
[202] va=feeca000 pte=739cd005: pfn=0x739cd user 
[203] va=feecb000 pte=739ce005: pfn=0x739ce user 
[205] va=feecd000 pte=739d0025: pfn=0x739d0 user 
[206] va=feece000 pte=739d1005: pfn=0x739d1 user 
[207] va=feecf000 pte=739d2005: pfn=0x739d2 user 
[208] va=feed0000 pte=739d3025: pfn=0x739d3 user 
[209] va=feed1000 pte=739d4025: pfn=0x739d4 user 
[210] va=feed2000 pte=739d5005: pfn=0x739d5 user 
[211] va=feed3000 pte=739d6025: pfn=0x739d6 user 
[212] va=feed4000 pte=739d7005: pfn=0x739d7 user 
[213] va=feed5000 pte=739d8005: pfn=0x739d8 user 
[214] va=feed6000 pte=739d9005: pfn=0x739d9 user 
[215] va=feed7000 pte=739da005: pfn=0x739da user 
[216] va=feed8000 pte=739db005: pfn=0x739db user 
[217] va=feed9000 pte=739dc005: pfn=0x739dc user 
[218] va=feeda000 pte=739dd005: pfn=0x739dd user 
[219] va=feedb000 pte=74ede025: pfn=0x74ede user 
[220] va=feedc000 pte=75f5f025: pfn=0x75f5f user 
[221] va=feedd000 pte=75ce0005: pfn=0x75ce0 user 
[222] va=feede000 pte=75de1005: pfn=0x75de1 user 
[223] va=feedf000 pte=75b62005: pfn=0x75b62 user 
[224] va=feee0000 pte=76ce3025: pfn=0x76ce3 user 
[225] va=feee1000 pte=76964005: pfn=0x76964 user 
[226] va=feee2000 pte=771e5005: pfn=0x771e5 user 
[230] va=feee6000 pte=755e9025: pfn=0x755e9 user 
[231] va=feee7000 pte=75d6a005: pfn=0x75d6a user 
[232] va=feee8000 pte=74f6b025: pfn=0x74f6b user 
[233] va=feee9000 pte=75dec005: pfn=0x75dec user 
[234] va=feeea000 pte=755ed025: pfn=0x755ed user 
[235] va=feeeb000 pte=75eee005: pfn=0x75eee user 
[236] va=feeec000 pte=31ef005: pfn=0x31ef user 
[237] va=feeed000 pte=756f0005: pfn=0x756f0 user 
[238] va=feeee000 pte=75271025: pfn=0x75271 user 
[239] va=feeef000 pte=772f2025: pfn=0x772f2 user 
[243] va=feef3000 pte=76376025: pfn=0x76376 user 
[244] va=feef4000 pte=74f77025: pfn=0x74f77 user 
[245] va=feef5000 pte=761f8005: pfn=0x761f8 user 
[246] va=feef6000 pte=76779005: pfn=0x76779 user 
[247] va=feef7000 pte=76c7a025: pfn=0x76c7a user 
[248] va=feef8000 pte=75afb025: pfn=0x75afb user 
[249] va=feef9000 pte=75afc005: pfn=0x75afc user 
[250] va=feefa000 pte=75dfd005: pfn=0x75dfd user 
[251] va=feefb000 pte=7547e025: pfn=0x7547e user 
[252] va=feefc000 pte=73b7f025: pfn=0x73b7f user 
[253] va=feefd000 pte=73c00025: pfn=0x73c00 user 
[254] va=feefe000 pte=75601025: pfn=0x75601 user 
[255] va=feeff000 pte=74b02025: pfn=0x74b02 user 
[256] va=fef00000 pte=75c83005: pfn=0x75c83 user 
[257] va=fef01000 pte=75204005: pfn=0x75204 user 
[258] va=fef02000 pte=75405005: pfn=0x75405 user 
[259] va=fef03000 pte=75586025: pfn=0x75586 user 
[260] va=fef04000 pte=76787025: pfn=0x76787 user 
[261] va=fef05000 pte=75208025: pfn=0x75208 user 
[262] va=fef06000 pte=75209025: pfn=0x75209 user 
[263] va=fef07000 pte=7678a025: pfn=0x7678a user 
[264] va=fef08000 pte=7528b005: pfn=0x7528b user 
[265] va=fef09000 pte=74c0c005: pfn=0x74c0c user 
[266] va=fef0a000 pte=7558d005: pfn=0x7558d user 
[267] va=fef0b000 pte=7680e005: pfn=0x7680e user 
[268] va=fef0c000 pte=7670f005: pfn=0x7670f user 
[269] va=fef0d000 pte=75a10005: pfn=0x75a10 user 
[271] va=fef0f000 pte=73912025: pfn=0x73912 user 
[272] va=fef10000 pte=73913005: pfn=0x73913 user 
[273] va=fef11000 pte=73914005: pfn=0x73914 user 
[274] va=fef12000 pte=73915005: pfn=0x73915 user 
[275] va=fef13000 pte=73916005: pfn=0x73916 user 
[276] va=fef14000 pte=73917005: pfn=0x73917 user 
[277] va=fef15000 pte=73918005: pfn=0x73918 user 
[278] va=fef16000 pte=73999005: pfn=0x73999 user 
[348] va=fef5c000 pte=739df025: pfn=0x739df user 
[349] va=fef5d000 pte=739e0025: pfn=0x739e0 user 
[350] va=fef5e000 pte=73be1025: pfn=0x73be1 user 
[351] va=fef5f000 pte=767e2005: pfn=0x767e2 user 
[352] va=fef60000 pte=75de3005: pfn=0x75de3 user 
[369] va=fef71000 pte=56402067: pfn=0x56402 user write    <----- writable (and written) pages restart from here
[370] va=fef72000 pte=6fc03067: pfn=0x6fc03 user write 
[371] va=fef73000 pte=50404067: pfn=0x50404 user write 
[372] va=fef74000 pte=64a05067: pfn=0x64a05 user write 
[373] va=fef75000 pte=61c06067: pfn=0x61c06 user write 
[374] va=fef76000 pte=5ce07067: pfn=0x5ce07 user write 
[375] va=fef77000 pte=50f88067: pfn=0x50f88 user write 
[376] va=fef78000 pte=5a789067: pfn=0x5a789 user write 
[377] va=fef79000 pte=5678a067: pfn=0x5678a user write 
[378] va=fef7a000 pte=5ca0b067: pfn=0x5ca0b user write 
[379] va=fef7b000 pte=4c58c067: pfn=0x4c58c user write 
[384] va=fef80000 pte=800000005aa11067: noexec pfn=0x5aa11 user write   <----- and another nonexec region, as per pmaps above
[400] va=fef90000 pte=800000006ff21067: noexec pfn=0x6ff21 user write 
[416] va=fefa0000 pte=66c31067: pfn=0x66c31 user write 
[432] va=fefb0000 pte=5cec1067: pfn=0x5cec1 user write 
[439] va=fefb7000 pte=7ae9f025: pfn=0x7ae9f user 
[440] va=fefb8000 pte=7bc20005: pfn=0x7bc20 user 
[441] va=fefb9000 pte=7af21005: pfn=0x7af21 user 
[442] va=fefba000 pte=7aea2025: pfn=0x7aea2 user 
[443] va=fefbb000 pte=7afa3025: pfn=0x7afa3 user 
[444] va=fefbc000 pte=7ce24025: pfn=0x7ce24 user 
[445] va=fefbd000 pte=7af25025: pfn=0x7af25 user 
[446] va=fefbe000 pte=7bc26025: pfn=0x7bc26 user 
[447] va=fefbf000 pte=7af27025: pfn=0x7af27 user 
[448] va=fefc0000 pte=7aea8025: pfn=0x7aea8 user 
[449] va=fefc1000 pte=7aa29025: pfn=0x7aa29 user 
[450] va=fefc2000 pte=7ae2a025: pfn=0x7ae2a user 
[451] va=fefc3000 pte=7942b025: pfn=0x7942b user 
[452] va=fefc4000 pte=798ac025: pfn=0x798ac user 
[453] va=fefc5000 pte=798ad025: pfn=0x798ad user 
[454] va=fefc6000 pte=7962e025: pfn=0x7962e user 
[455] va=fefc7000 pte=7ac2f025: pfn=0x7ac2f user 
[456] va=fefc8000 pte=79630025: pfn=0x79630 user 
[457] va=fefc9000 pte=79631025: pfn=0x79631 user 
[458] va=fefca000 pte=79832025: pfn=0x79832 user 
[459] va=fefcb000 pte=79833025: pfn=0x79833 user 
[460] va=fefcc000 pte=79834025: pfn=0x79834 user 
[461] va=fefcd000 pte=79635025: pfn=0x79635 user 
[462] va=fefce000 pte=796b6025: pfn=0x796b6 user 
[463] va=fefcf000 pte=79837025: pfn=0x79837 user 
[464] va=fefd0000 pte=7ae38005: pfn=0x7ae38 user 
[465] va=fefd1000 pte=7acb9025: pfn=0x7acb9 user 
[466] va=fefd2000 pte=7983a025: pfn=0x7983a user 
[467] va=fefd3000 pte=7aa3b005: pfn=0x7aa3b user 
[468] va=fefd4000 pte=7aebc025: pfn=0x7aebc user 
[469] va=fefd5000 pte=7aa3d025: pfn=0x7aa3d user 
[470] va=fefd6000 pte=7bc3e025: pfn=0x7bc3e user 
[471] va=fefd7000 pte=7bc3f005: pfn=0x7bc3f user 
[472] va=fefd8000 pte=7bb40005: pfn=0x7bb40 user 
[473] va=fefd9000 pte=7aec1025: pfn=0x7aec1 user 
[474] va=fefda000 pte=7bb42025: pfn=0x7bb42 user 
[475] va=fefdb000 pte=7bb43025: pfn=0x7bb43 user 
[476] va=fefdc000 pte=7adc4025: pfn=0x7adc4 user 
[477] va=fefdd000 pte=7bbc5025: pfn=0x7bbc5 user 
[478] va=fefde000 pte=7bbc6005: pfn=0x7bbc6 user 
[479] va=fefdf000 pte=7aa47005: pfn=0x7aa47 user 
[480] va=fefe0000 pte=7bb48005: pfn=0x7bb48 user 
[481] va=fefe1000 pte=7bb49005: pfn=0x7bb49 user 
[482] va=fefe2000 pte=79d4a025: pfn=0x79d4a user 
[483] va=fefe3000 pte=7accb025: pfn=0x7accb user 
[484] va=fefe4000 pte=79d4c025: pfn=0x79d4c user 
[485] va=fefe5000 pte=7bb4d025: pfn=0x7bb4d user 
[486] va=fefe6000 pte=454e025: pfn=0x454e user 
[487] va=fefe7000 pte=79d4f025: pfn=0x79d4f user 
[488] va=fefe8000 pte=3750005: pfn=0x3750 user 
[489] va=fefe9000 pte=37d1005: pfn=0x37d1 user 
[490] va=fefea000 pte=4752005: pfn=0x4752 user 
[507] va=feffb000 pte=5f50c067: pfn=0x5f50c user write 
[508] va=feffc000 pte=5a70d067: pfn=0x5a70d user write 
[509] va=feffd000 pte=5ac0e067: pfn=0x5ac0e user write 

And so it goes.

But wait, there's more! With MDB's DCMDs, we can find in which process
a physical page is mapped, and at which virtual address!

> ::dcmds ! grep -i pfn
ptable                   - Given PFN, dump contents of a page table
report_maps              - Given PFN, report mappings / page table usage      <--- this one
mfntopfn                 - convert hypervisor machine page to physical page
pfntomfn                 - convert physical page to hypervisor machine page
vatopfn                  - translate address to physical page

We'll track one of those rare NX-protected pages above:

> 0x05aa11::report_maps
hat=ffffff0157c3d230 maps addr=fef80000

Important: 
How does this command know which process mapped that page? 

And we can get back to the process descriptor and the process name, of course: 

> ffffff0157c3d230::print 'struct hat' hat_as | ::print 'struct as' a_proc | ::print proc_t ! grep comm
        u_comm = [ "mmap" ]

Now read textbook's 12.3.6.3 and figure how hment_t and page_t's p_mapping figure into this!

Note the HTABLE_HASH (http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/i86pc/vm/htable.h#96).
Where is it used and why?

96 /*
97  * The htable hash table hashing function.  The 28 is so that high
98  * order bits are include in the hash index to skew the wrap
99  * around of addresses. Even though the hash buckets are stored per
100 * hat we include the value of hat pointer in the hash function so
101 * that the secondary hash for the htable mutex winds up begin different in
102 * every address space.
103 */
104#define	HTABLE_HASH(hat, va, lvl)					\
105		((((va) >> LEVEL_SHIFT(1)) + ((va) >> 28) + (lvl) +			\
106		((uintptr_t)(hat) >> 4)) & ((hat)->hat_num_hash - 1))