blob: 34ddb1b1e2eec63eb348a2758f08a1ce1e240e17 [file] [log] [blame]
dan7c246102010-04-12 19:00:29 +00001/*
drh7ed91f22010-04-29 22:34:07 +00002** 2010 February 1
3**
4** The author disclaims copyright to this source code. In place of
5** a legal notice, here is a blessing:
6**
7** May you do good and not evil.
8** May you find forgiveness for yourself and forgive others.
9** May you share freely, never taking more than you give.
10**
11*************************************************************************
12**
drh027a1282010-05-19 01:53:53 +000013** This file contains the implementation of a write-ahead log (WAL) used in
14** "journal_mode=WAL" mode.
drh29d4dbe2010-05-18 23:29:52 +000015**
drh7ed91f22010-04-29 22:34:07 +000016** WRITE-AHEAD LOG (WAL) FILE FORMAT
dan97a31352010-04-16 13:59:31 +000017**
drh7e263722010-05-20 21:21:09 +000018** A WAL file consists of a header followed by zero or more "frames".
drh027a1282010-05-19 01:53:53 +000019** Each frame records the revised content of a single page from the
drh29d4dbe2010-05-18 23:29:52 +000020** database file. All changes to the database are recorded by writing
21** frames into the WAL. Transactions commit when a frame is written that
22** contains a commit marker. A single WAL can and usually does record
23** multiple transactions. Periodically, the content of the WAL is
24** transferred back into the database file in an operation called a
25** "checkpoint".
26**
27** A single WAL file can be used multiple times. In other words, the
drh027a1282010-05-19 01:53:53 +000028** WAL can fill up with frames and then be checkpointed and then new
drh29d4dbe2010-05-18 23:29:52 +000029** frames can overwrite the old ones. A WAL always grows from beginning
30** toward the end. Checksums and counters attached to each frame are
31** used to determine which frames within the WAL are valid and which
32** are leftovers from prior checkpoints.
33**
drhcd285082010-06-23 22:00:35 +000034** The WAL header is 32 bytes in size and consists of the following eight
dan97a31352010-04-16 13:59:31 +000035** big-endian 32-bit unsigned integer values:
36**
drh1b78eaf2010-05-25 13:40:03 +000037** 0: Magic number. 0x377f0682 or 0x377f0683
drh23ea97b2010-05-20 16:45:58 +000038** 4: File format version. Currently 3007000
39** 8: Database page size. Example: 1024
40** 12: Checkpoint sequence number
drh7e263722010-05-20 21:21:09 +000041** 16: Salt-1, random integer incremented with each checkpoint
42** 20: Salt-2, a different random integer changing with each ckpt
dan10f5a502010-06-23 15:55:43 +000043** 24: Checksum-1 (first part of checksum for first 24 bytes of header).
44** 28: Checksum-2 (second part of checksum for first 24 bytes of header).
dan97a31352010-04-16 13:59:31 +000045**
drh23ea97b2010-05-20 16:45:58 +000046** Immediately following the wal-header are zero or more frames. Each
47** frame consists of a 24-byte frame-header followed by a <page-size> bytes
drhcd285082010-06-23 22:00:35 +000048** of page data. The frame-header is six big-endian 32-bit unsigned
dan97a31352010-04-16 13:59:31 +000049** integer values, as follows:
50**
dan3de777f2010-04-17 12:31:37 +000051** 0: Page number.
52** 4: For commit records, the size of the database image in pages
dan97a31352010-04-16 13:59:31 +000053** after the commit. For all other records, zero.
drh7e263722010-05-20 21:21:09 +000054** 8: Salt-1 (copied from the header)
55** 12: Salt-2 (copied from the header)
drh23ea97b2010-05-20 16:45:58 +000056** 16: Checksum-1.
57** 20: Checksum-2.
drh29d4dbe2010-05-18 23:29:52 +000058**
drh7e263722010-05-20 21:21:09 +000059** A frame is considered valid if and only if the following conditions are
60** true:
61**
62** (1) The salt-1 and salt-2 values in the frame-header match
63** salt values in the wal-header
64**
65** (2) The checksum values in the final 8 bytes of the frame-header
drh1b78eaf2010-05-25 13:40:03 +000066** exactly match the checksum computed consecutively on the
67** WAL header and the first 8 bytes and the content of all frames
68** up to and including the current frame.
69**
70** The checksum is computed using 32-bit big-endian integers if the
71** magic number in the first 4 bytes of the WAL is 0x377f0683 and it
72** is computed using little-endian if the magic number is 0x377f0682.
drh51b21b12010-05-25 15:53:31 +000073** The checksum values are always stored in the frame header in a
74** big-endian format regardless of which byte order is used to compute
75** the checksum. The checksum is computed by interpreting the input as
76** an even number of unsigned 32-bit integers: x[0] through x[N]. The
drhffca4302010-06-15 11:21:54 +000077** algorithm used for the checksum is as follows:
drh51b21b12010-05-25 15:53:31 +000078**
79** for i from 0 to n-1 step 2:
80** s0 += x[i] + s1;
81** s1 += x[i+1] + s0;
82** endfor
drh7e263722010-05-20 21:21:09 +000083**
drhcd285082010-06-23 22:00:35 +000084** Note that s0 and s1 are both weighted checksums using fibonacci weights
85** in reverse order (the largest fibonacci weight occurs on the first element
86** of the sequence being summed.) The s1 value spans all 32-bit
87** terms of the sequence whereas s0 omits the final term.
88**
drh7e263722010-05-20 21:21:09 +000089** On a checkpoint, the WAL is first VFS.xSync-ed, then valid content of the
90** WAL is transferred into the database, then the database is VFS.xSync-ed.
drhffca4302010-06-15 11:21:54 +000091** The VFS.xSync operations serve as write barriers - all writes launched
drh7e263722010-05-20 21:21:09 +000092** before the xSync must complete before any write that launches after the
93** xSync begins.
94**
95** After each checkpoint, the salt-1 value is incremented and the salt-2
96** value is randomized. This prevents old and new frames in the WAL from
97** being considered valid at the same time and being checkpointing together
98** following a crash.
99**
drh29d4dbe2010-05-18 23:29:52 +0000100** READER ALGORITHM
101**
102** To read a page from the database (call it page number P), a reader
103** first checks the WAL to see if it contains page P. If so, then the
drh73b64e42010-05-30 19:55:15 +0000104** last valid instance of page P that is a followed by a commit frame
105** or is a commit frame itself becomes the value read. If the WAL
106** contains no copies of page P that are valid and which are a commit
107** frame or are followed by a commit frame, then page P is read from
108** the database file.
drh29d4dbe2010-05-18 23:29:52 +0000109**
drh73b64e42010-05-30 19:55:15 +0000110** To start a read transaction, the reader records the index of the last
111** valid frame in the WAL. The reader uses this recorded "mxFrame" value
112** for all subsequent read operations. New transactions can be appended
113** to the WAL, but as long as the reader uses its original mxFrame value
114** and ignores the newly appended content, it will see a consistent snapshot
115** of the database from a single point in time. This technique allows
116** multiple concurrent readers to view different versions of the database
117** content simultaneously.
118**
119** The reader algorithm in the previous paragraphs works correctly, but
drh29d4dbe2010-05-18 23:29:52 +0000120** because frames for page P can appear anywhere within the WAL, the
drh027a1282010-05-19 01:53:53 +0000121** reader has to scan the entire WAL looking for page P frames. If the
drh29d4dbe2010-05-18 23:29:52 +0000122** WAL is large (multiple megabytes is typical) that scan can be slow,
drh027a1282010-05-19 01:53:53 +0000123** and read performance suffers. To overcome this problem, a separate
124** data structure called the wal-index is maintained to expedite the
drh29d4dbe2010-05-18 23:29:52 +0000125** search for frames of a particular page.
126**
127** WAL-INDEX FORMAT
128**
129** Conceptually, the wal-index is shared memory, though VFS implementations
130** might choose to implement the wal-index using a mmapped file. Because
131** the wal-index is shared memory, SQLite does not support journal_mode=WAL
132** on a network filesystem. All users of the database must be able to
133** share memory.
134**
135** The wal-index is transient. After a crash, the wal-index can (and should
136** be) reconstructed from the original WAL file. In fact, the VFS is required
137** to either truncate or zero the header of the wal-index when the last
138** connection to it closes. Because the wal-index is transient, it can
139** use an architecture-specific format; it does not have to be cross-platform.
140** Hence, unlike the database and WAL file formats which store all values
141** as big endian, the wal-index can store multi-byte values in the native
142** byte order of the host computer.
143**
144** The purpose of the wal-index is to answer this question quickly: Given
145** a page number P, return the index of the last frame for page P in the WAL,
146** or return NULL if there are no frames for page P in the WAL.
147**
148** The wal-index consists of a header region, followed by an one or
149** more index blocks.
150**
drh027a1282010-05-19 01:53:53 +0000151** The wal-index header contains the total number of frames within the WAL
danad3cadd2010-06-14 11:49:26 +0000152** in the the mxFrame field.
153**
154** Each index block except for the first contains information on
155** HASHTABLE_NPAGE frames. The first index block contains information on
156** HASHTABLE_NPAGE_ONE frames. The values of HASHTABLE_NPAGE_ONE and
157** HASHTABLE_NPAGE are selected so that together the wal-index header and
158** first index block are the same size as all other index blocks in the
159** wal-index.
160**
161** Each index block contains two sections, a page-mapping that contains the
162** database page number associated with each wal frame, and a hash-table
drhffca4302010-06-15 11:21:54 +0000163** that allows readers to query an index block for a specific page number.
danad3cadd2010-06-14 11:49:26 +0000164** The page-mapping is an array of HASHTABLE_NPAGE (or HASHTABLE_NPAGE_ONE
165** for the first index block) 32-bit page numbers. The first entry in the
166** first index-block contains the database page number corresponding to the
167** first frame in the WAL file. The first entry in the second index block
168** in the WAL file corresponds to the (HASHTABLE_NPAGE_ONE+1)th frame in
169** the log, and so on.
170**
171** The last index block in a wal-index usually contains less than the full
172** complement of HASHTABLE_NPAGE (or HASHTABLE_NPAGE_ONE) page-numbers,
173** depending on the contents of the WAL file. This does not change the
174** allocated size of the page-mapping array - the page-mapping array merely
175** contains unused entries.
drh027a1282010-05-19 01:53:53 +0000176**
177** Even without using the hash table, the last frame for page P
danad3cadd2010-06-14 11:49:26 +0000178** can be found by scanning the page-mapping sections of each index block
drh027a1282010-05-19 01:53:53 +0000179** starting with the last index block and moving toward the first, and
180** within each index block, starting at the end and moving toward the
181** beginning. The first entry that equals P corresponds to the frame
182** holding the content for that page.
183**
184** The hash table consists of HASHTABLE_NSLOT 16-bit unsigned integers.
185** HASHTABLE_NSLOT = 2*HASHTABLE_NPAGE, and there is one entry in the
186** hash table for each page number in the mapping section, so the hash
187** table is never more than half full. The expected number of collisions
188** prior to finding a match is 1. Each entry of the hash table is an
189** 1-based index of an entry in the mapping section of the same
190** index block. Let K be the 1-based index of the largest entry in
191** the mapping section. (For index blocks other than the last, K will
192** always be exactly HASHTABLE_NPAGE (4096) and for the last index block
193** K will be (mxFrame%HASHTABLE_NPAGE).) Unused slots of the hash table
drh73b64e42010-05-30 19:55:15 +0000194** contain a value of 0.
drh027a1282010-05-19 01:53:53 +0000195**
196** To look for page P in the hash table, first compute a hash iKey on
197** P as follows:
198**
199** iKey = (P * 383) % HASHTABLE_NSLOT
200**
201** Then start scanning entries of the hash table, starting with iKey
202** (wrapping around to the beginning when the end of the hash table is
203** reached) until an unused hash slot is found. Let the first unused slot
204** be at index iUnused. (iUnused might be less than iKey if there was
205** wrap-around.) Because the hash table is never more than half full,
206** the search is guaranteed to eventually hit an unused entry. Let
207** iMax be the value between iKey and iUnused, closest to iUnused,
208** where aHash[iMax]==P. If there is no iMax entry (if there exists
209** no hash slot such that aHash[i]==p) then page P is not in the
210** current index block. Otherwise the iMax-th mapping entry of the
211** current index block corresponds to the last entry that references
212** page P.
213**
214** A hash search begins with the last index block and moves toward the
215** first index block, looking for entries corresponding to page P. On
216** average, only two or three slots in each index block need to be
217** examined in order to either find the last entry for page P, or to
218** establish that no such entry exists in the block. Each index block
219** holds over 4000 entries. So two or three index blocks are sufficient
220** to cover a typical 10 megabyte WAL file, assuming 1K pages. 8 or 10
221** comparisons (on average) suffice to either locate a frame in the
222** WAL or to establish that the frame does not exist in the WAL. This
223** is much faster than scanning the entire 10MB WAL.
224**
225** Note that entries are added in order of increasing K. Hence, one
226** reader might be using some value K0 and a second reader that started
227** at a later time (after additional transactions were added to the WAL
228** and to the wal-index) might be using a different value K1, where K1>K0.
229** Both readers can use the same hash table and mapping section to get
230** the correct result. There may be entries in the hash table with
231** K>K0 but to the first reader, those entries will appear to be unused
232** slots in the hash table and so the first reader will get an answer as
233** if no values greater than K0 had ever been inserted into the hash table
234** in the first place - which is what reader one wants. Meanwhile, the
235** second reader using K1 will see additional values that were inserted
236** later, which is exactly what reader two wants.
237**
dan6f150142010-05-21 15:31:56 +0000238** When a rollback occurs, the value of K is decreased. Hash table entries
239** that correspond to frames greater than the new K value are removed
240** from the hash table at this point.
dan97a31352010-04-16 13:59:31 +0000241*/
drh29d4dbe2010-05-18 23:29:52 +0000242#ifndef SQLITE_OMIT_WAL
dan97a31352010-04-16 13:59:31 +0000243
drh29d4dbe2010-05-18 23:29:52 +0000244#include "wal.h"
245
drh73b64e42010-05-30 19:55:15 +0000246/*
drhc74c3332010-05-31 12:15:19 +0000247** Trace output macros
248*/
drhc74c3332010-05-31 12:15:19 +0000249#if defined(SQLITE_TEST) && defined(SQLITE_DEBUG)
drh15d68092010-05-31 16:56:14 +0000250int sqlite3WalTrace = 0;
drhc74c3332010-05-31 12:15:19 +0000251# define WALTRACE(X) if(sqlite3WalTrace) sqlite3DebugPrintf X
252#else
253# define WALTRACE(X)
254#endif
255
dan10f5a502010-06-23 15:55:43 +0000256/*
257** The maximum (and only) versions of the wal and wal-index formats
258** that may be interpreted by this version of SQLite.
259**
260** If a client begins recovering a WAL file and finds that (a) the checksum
261** values in the wal-header are correct and (b) the version field is not
262** WAL_MAX_VERSION, recovery fails and SQLite returns SQLITE_CANTOPEN.
263**
264** Similarly, if a client successfully reads a wal-index header (i.e. the
265** checksum test is successful) and finds that the version field is not
266** WALINDEX_MAX_VERSION, then no read-transaction is opened and SQLite
267** returns SQLITE_CANTOPEN.
268*/
269#define WAL_MAX_VERSION 3007000
270#define WALINDEX_MAX_VERSION 3007000
drhc74c3332010-05-31 12:15:19 +0000271
272/*
drh73b64e42010-05-30 19:55:15 +0000273** Indices of various locking bytes. WAL_NREADER is the number
274** of available reader locks and should be at least 3.
275*/
276#define WAL_WRITE_LOCK 0
277#define WAL_ALL_BUT_WRITE 1
278#define WAL_CKPT_LOCK 1
279#define WAL_RECOVER_LOCK 2
280#define WAL_READ_LOCK(I) (3+(I))
281#define WAL_NREADER (SQLITE_SHM_NLOCK-3)
282
dan97a31352010-04-16 13:59:31 +0000283
drh7ed91f22010-04-29 22:34:07 +0000284/* Object declarations */
285typedef struct WalIndexHdr WalIndexHdr;
286typedef struct WalIterator WalIterator;
drh73b64e42010-05-30 19:55:15 +0000287typedef struct WalCkptInfo WalCkptInfo;
dan7c246102010-04-12 19:00:29 +0000288
289
290/*
drh286a2882010-05-20 23:51:06 +0000291** The following object holds a copy of the wal-index header content.
292**
293** The actual header in the wal-index consists of two copies of this
294** object.
drh9b78f792010-08-14 21:21:24 +0000295**
296** The szPage value can be any power of 2 between 512 and 32768, inclusive.
297** Or it can be 1 to represent a 65536-byte page. The latter case was
298** added in 3.7.1 when support for 64K pages was added.
dan7c246102010-04-12 19:00:29 +0000299*/
drh7ed91f22010-04-29 22:34:07 +0000300struct WalIndexHdr {
dan10f5a502010-06-23 15:55:43 +0000301 u32 iVersion; /* Wal-index version */
302 u32 unused; /* Unused (padding) field */
dan71d89912010-05-24 13:57:42 +0000303 u32 iChange; /* Counter incremented each transaction */
drh4b82c382010-05-31 18:24:19 +0000304 u8 isInit; /* 1 when initialized */
305 u8 bigEndCksum; /* True if checksums in WAL are big-endian */
drh9b78f792010-08-14 21:21:24 +0000306 u16 szPage; /* Database page size in bytes. 1==64K */
dand0aa3422010-05-31 16:41:53 +0000307 u32 mxFrame; /* Index of last valid frame in the WAL */
dan71d89912010-05-24 13:57:42 +0000308 u32 nPage; /* Size of database in pages */
309 u32 aFrameCksum[2]; /* Checksum of last frame in log */
310 u32 aSalt[2]; /* Two salt values copied from WAL header */
311 u32 aCksum[2]; /* Checksum over all prior fields */
dan7c246102010-04-12 19:00:29 +0000312};
313
drh73b64e42010-05-30 19:55:15 +0000314/*
315** A copy of the following object occurs in the wal-index immediately
316** following the second copy of the WalIndexHdr. This object stores
317** information used by checkpoint.
318**
319** nBackfill is the number of frames in the WAL that have been written
320** back into the database. (We call the act of moving content from WAL to
321** database "backfilling".) The nBackfill number is never greater than
322** WalIndexHdr.mxFrame. nBackfill can only be increased by threads
323** holding the WAL_CKPT_LOCK lock (which includes a recovery thread).
324** However, a WAL_WRITE_LOCK thread can move the value of nBackfill from
325** mxFrame back to zero when the WAL is reset.
326**
327** There is one entry in aReadMark[] for each reader lock. If a reader
328** holds read-lock K, then the value in aReadMark[K] is no greater than
drhdb7f6472010-06-09 14:45:12 +0000329** the mxFrame for that reader. The value READMARK_NOT_USED (0xffffffff)
330** for any aReadMark[] means that entry is unused. aReadMark[0] is
331** a special case; its value is never used and it exists as a place-holder
332** to avoid having to offset aReadMark[] indexs by one. Readers holding
333** WAL_READ_LOCK(0) always ignore the entire WAL and read all content
334** directly from the database.
drh73b64e42010-05-30 19:55:15 +0000335**
336** The value of aReadMark[K] may only be changed by a thread that
337** is holding an exclusive lock on WAL_READ_LOCK(K). Thus, the value of
338** aReadMark[K] cannot changed while there is a reader is using that mark
339** since the reader will be holding a shared lock on WAL_READ_LOCK(K).
340**
341** The checkpointer may only transfer frames from WAL to database where
342** the frame numbers are less than or equal to every aReadMark[] that is
343** in use (that is, every aReadMark[j] for which there is a corresponding
344** WAL_READ_LOCK(j)). New readers (usually) pick the aReadMark[] with the
345** largest value and will increase an unused aReadMark[] to mxFrame if there
346** is not already an aReadMark[] equal to mxFrame. The exception to the
347** previous sentence is when nBackfill equals mxFrame (meaning that everything
348** in the WAL has been backfilled into the database) then new readers
349** will choose aReadMark[0] which has value 0 and hence such reader will
350** get all their all content directly from the database file and ignore
351** the WAL.
352**
353** Writers normally append new frames to the end of the WAL. However,
354** if nBackfill equals mxFrame (meaning that all WAL content has been
355** written back into the database) and if no readers are using the WAL
356** (in other words, if there are no WAL_READ_LOCK(i) where i>0) then
357** the writer will first "reset" the WAL back to the beginning and start
358** writing new content beginning at frame 1.
359**
360** We assume that 32-bit loads are atomic and so no locks are needed in
361** order to read from any aReadMark[] entries.
362*/
363struct WalCkptInfo {
364 u32 nBackfill; /* Number of WAL frames backfilled into DB */
365 u32 aReadMark[WAL_NREADER]; /* Reader marks */
366};
drhdb7f6472010-06-09 14:45:12 +0000367#define READMARK_NOT_USED 0xffffffff
drh73b64e42010-05-30 19:55:15 +0000368
369
drh7e263722010-05-20 21:21:09 +0000370/* A block of WALINDEX_LOCK_RESERVED bytes beginning at
371** WALINDEX_LOCK_OFFSET is reserved for locks. Since some systems
372** only support mandatory file-locks, we do not read or write data
373** from the region of the file on which locks are applied.
danff207012010-04-24 04:49:15 +0000374*/
drh73b64e42010-05-30 19:55:15 +0000375#define WALINDEX_LOCK_OFFSET (sizeof(WalIndexHdr)*2 + sizeof(WalCkptInfo))
376#define WALINDEX_LOCK_RESERVED 16
drh026ac282010-05-26 15:06:38 +0000377#define WALINDEX_HDR_SIZE (WALINDEX_LOCK_OFFSET+WALINDEX_LOCK_RESERVED)
dan7c246102010-04-12 19:00:29 +0000378
drh7ed91f22010-04-29 22:34:07 +0000379/* Size of header before each frame in wal */
drh23ea97b2010-05-20 16:45:58 +0000380#define WAL_FRAME_HDRSIZE 24
danff207012010-04-24 04:49:15 +0000381
dan10f5a502010-06-23 15:55:43 +0000382/* Size of write ahead log header, including checksum. */
383/* #define WAL_HDRSIZE 24 */
384#define WAL_HDRSIZE 32
dan97a31352010-04-16 13:59:31 +0000385
danb8fd6c22010-05-24 10:39:36 +0000386/* WAL magic value. Either this value, or the same value with the least
387** significant bit also set (WAL_MAGIC | 0x00000001) is stored in 32-bit
388** big-endian format in the first 4 bytes of a WAL file.
389**
390** If the LSB is set, then the checksums for each frame within the WAL
391** file are calculated by treating all data as an array of 32-bit
392** big-endian words. Otherwise, they are calculated by interpreting
393** all data as 32-bit little-endian words.
394*/
395#define WAL_MAGIC 0x377f0682
396
dan97a31352010-04-16 13:59:31 +0000397/*
drh7ed91f22010-04-29 22:34:07 +0000398** Return the offset of frame iFrame in the write-ahead log file,
drh6e810962010-05-19 17:49:50 +0000399** assuming a database page size of szPage bytes. The offset returned
drh7ed91f22010-04-29 22:34:07 +0000400** is to the start of the write-ahead log frame-header.
dan97a31352010-04-16 13:59:31 +0000401*/
drh6e810962010-05-19 17:49:50 +0000402#define walFrameOffset(iFrame, szPage) ( \
danbd0e9072010-07-07 09:48:44 +0000403 WAL_HDRSIZE + ((iFrame)-1)*(i64)((szPage)+WAL_FRAME_HDRSIZE) \
dan97a31352010-04-16 13:59:31 +0000404)
dan7c246102010-04-12 19:00:29 +0000405
406/*
drh7ed91f22010-04-29 22:34:07 +0000407** An open write-ahead log file is represented by an instance of the
408** following object.
dance4f05f2010-04-22 19:14:13 +0000409*/
drh7ed91f22010-04-29 22:34:07 +0000410struct Wal {
drh73b64e42010-05-30 19:55:15 +0000411 sqlite3_vfs *pVfs; /* The VFS used to create pDbFd */
drhd9e5c4f2010-05-12 18:01:39 +0000412 sqlite3_file *pDbFd; /* File handle for the database file */
413 sqlite3_file *pWalFd; /* File handle for WAL file */
drh7ed91f22010-04-29 22:34:07 +0000414 u32 iCallback; /* Value to pass to log callback (or 0) */
drh85a83752011-05-16 21:00:27 +0000415 i64 mxWalSize; /* Truncate WAL to this size upon reset */
dan13a3cb82010-06-11 19:04:21 +0000416 int nWiData; /* Size of array apWiData */
drh88f975a2011-12-16 19:34:36 +0000417 int szFirstBlock; /* Size of first block written to WAL file */
dan13a3cb82010-06-11 19:04:21 +0000418 volatile u32 **apWiData; /* Pointer to wal-index content in memory */
drhb2eced52010-08-12 02:41:12 +0000419 u32 szPage; /* Database page size */
drh73b64e42010-05-30 19:55:15 +0000420 i16 readLock; /* Which read lock is being held. -1 for none */
drh4eb02a42011-12-16 21:26:26 +0000421 u8 syncFlags; /* Flags to use to sync header writes */
dan55437592010-05-11 12:19:26 +0000422 u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */
drh73b64e42010-05-30 19:55:15 +0000423 u8 writeLock; /* True if in a write transaction */
424 u8 ckptLock; /* True if holding a checkpoint lock */
drh66dfec8b2011-06-01 20:01:49 +0000425 u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */
danf60b7f32011-12-16 13:24:27 +0000426 u8 truncateOnCommit; /* True to truncate WAL file on commit */
drh4eb02a42011-12-16 21:26:26 +0000427 u8 noSyncHeader; /* Avoid WAL header fsyncs if true */
drh73b64e42010-05-30 19:55:15 +0000428 WalIndexHdr hdr; /* Wal-index header for current transaction */
dan3e875ef2010-07-05 19:03:35 +0000429 const char *zWalName; /* Name of WAL file */
drh7e263722010-05-20 21:21:09 +0000430 u32 nCkpt; /* Checkpoint sequence counter in the wal-header */
drhaab4c022010-06-02 14:45:51 +0000431#ifdef SQLITE_DEBUG
432 u8 lockError; /* True if a locking error has occurred */
433#endif
dan7c246102010-04-12 19:00:29 +0000434};
435
drh73b64e42010-05-30 19:55:15 +0000436/*
dan8c408002010-11-01 17:38:24 +0000437** Candidate values for Wal.exclusiveMode.
438*/
439#define WAL_NORMAL_MODE 0
440#define WAL_EXCLUSIVE_MODE 1
441#define WAL_HEAPMEMORY_MODE 2
442
443/*
drh66dfec8b2011-06-01 20:01:49 +0000444** Possible values for WAL.readOnly
445*/
446#define WAL_RDWR 0 /* Normal read/write connection */
447#define WAL_RDONLY 1 /* The WAL file is readonly */
448#define WAL_SHM_RDONLY 2 /* The SHM file is readonly */
449
450/*
dan067f3162010-06-14 10:30:12 +0000451** Each page of the wal-index mapping contains a hash-table made up of
452** an array of HASHTABLE_NSLOT elements of the following type.
453*/
454typedef u16 ht_slot;
455
456/*
danad3cadd2010-06-14 11:49:26 +0000457** This structure is used to implement an iterator that loops through
458** all frames in the WAL in database page order. Where two or more frames
459** correspond to the same database page, the iterator visits only the
460** frame most recently written to the WAL (in other words, the frame with
461** the largest index).
462**
463** The internals of this structure are only accessed by:
464**
465** walIteratorInit() - Create a new iterator,
466** walIteratorNext() - Step an iterator,
467** walIteratorFree() - Free an iterator.
468**
469** This functionality is used by the checkpoint code (see walCheckpoint()).
470*/
471struct WalIterator {
472 int iPrior; /* Last result returned from the iterator */
drhd9c9b782010-12-15 21:02:06 +0000473 int nSegment; /* Number of entries in aSegment[] */
danad3cadd2010-06-14 11:49:26 +0000474 struct WalSegment {
475 int iNext; /* Next slot in aIndex[] not yet returned */
476 ht_slot *aIndex; /* i0, i1, i2... such that aPgno[iN] ascend */
477 u32 *aPgno; /* Array of page numbers. */
drhd9c9b782010-12-15 21:02:06 +0000478 int nEntry; /* Nr. of entries in aPgno[] and aIndex[] */
danad3cadd2010-06-14 11:49:26 +0000479 int iZero; /* Frame number associated with aPgno[0] */
drhd9c9b782010-12-15 21:02:06 +0000480 } aSegment[1]; /* One for every 32KB page in the wal-index */
danad3cadd2010-06-14 11:49:26 +0000481};
482
483/*
dan13a3cb82010-06-11 19:04:21 +0000484** Define the parameters of the hash tables in the wal-index file. There
485** is a hash-table following every HASHTABLE_NPAGE page numbers in the
486** wal-index.
487**
488** Changing any of these constants will alter the wal-index format and
489** create incompatibilities.
490*/
dan067f3162010-06-14 10:30:12 +0000491#define HASHTABLE_NPAGE 4096 /* Must be power of 2 */
dan13a3cb82010-06-11 19:04:21 +0000492#define HASHTABLE_HASH_1 383 /* Should be prime */
493#define HASHTABLE_NSLOT (HASHTABLE_NPAGE*2) /* Must be a power of 2 */
dan13a3cb82010-06-11 19:04:21 +0000494
danad3cadd2010-06-14 11:49:26 +0000495/*
496** The block of page numbers associated with the first hash-table in a
dan13a3cb82010-06-11 19:04:21 +0000497** wal-index is smaller than usual. This is so that there is a complete
498** hash-table on each aligned 32KB page of the wal-index.
499*/
dan067f3162010-06-14 10:30:12 +0000500#define HASHTABLE_NPAGE_ONE (HASHTABLE_NPAGE - (WALINDEX_HDR_SIZE/sizeof(u32)))
dan13a3cb82010-06-11 19:04:21 +0000501
dan067f3162010-06-14 10:30:12 +0000502/* The wal-index is divided into pages of WALINDEX_PGSZ bytes each. */
503#define WALINDEX_PGSZ ( \
504 sizeof(ht_slot)*HASHTABLE_NSLOT + HASHTABLE_NPAGE*sizeof(u32) \
505)
dan13a3cb82010-06-11 19:04:21 +0000506
507/*
508** Obtain a pointer to the iPage'th page of the wal-index. The wal-index
dan067f3162010-06-14 10:30:12 +0000509** is broken into pages of WALINDEX_PGSZ bytes. Wal-index pages are
dan13a3cb82010-06-11 19:04:21 +0000510** numbered from zero.
511**
512** If this call is successful, *ppPage is set to point to the wal-index
513** page and SQLITE_OK is returned. If an error (an OOM or VFS error) occurs,
514** then an SQLite error code is returned and *ppPage is set to 0.
515*/
516static int walIndexPage(Wal *pWal, int iPage, volatile u32 **ppPage){
517 int rc = SQLITE_OK;
518
519 /* Enlarge the pWal->apWiData[] array if required */
520 if( pWal->nWiData<=iPage ){
drh519426a2010-07-09 03:19:07 +0000521 int nByte = sizeof(u32*)*(iPage+1);
dan13a3cb82010-06-11 19:04:21 +0000522 volatile u32 **apNew;
shaneh8a300f82010-07-02 18:15:31 +0000523 apNew = (volatile u32 **)sqlite3_realloc((void *)pWal->apWiData, nByte);
dan13a3cb82010-06-11 19:04:21 +0000524 if( !apNew ){
525 *ppPage = 0;
526 return SQLITE_NOMEM;
527 }
drh519426a2010-07-09 03:19:07 +0000528 memset((void*)&apNew[pWal->nWiData], 0,
529 sizeof(u32*)*(iPage+1-pWal->nWiData));
dan13a3cb82010-06-11 19:04:21 +0000530 pWal->apWiData = apNew;
531 pWal->nWiData = iPage+1;
532 }
533
534 /* Request a pointer to the required page from the VFS */
535 if( pWal->apWiData[iPage]==0 ){
dan8c408002010-11-01 17:38:24 +0000536 if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){
537 pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ);
538 if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM;
539 }else{
540 rc = sqlite3OsShmMap(pWal->pDbFd, iPage, WALINDEX_PGSZ,
541 pWal->writeLock, (void volatile **)&pWal->apWiData[iPage]
542 );
drh66dfec8b2011-06-01 20:01:49 +0000543 if( rc==SQLITE_READONLY ){
544 pWal->readOnly |= WAL_SHM_RDONLY;
545 rc = SQLITE_OK;
dan4edc6bf2011-05-10 17:31:29 +0000546 }
dan8c408002010-11-01 17:38:24 +0000547 }
dan13a3cb82010-06-11 19:04:21 +0000548 }
danb6d2f9c2011-05-11 14:57:33 +0000549
drh66dfec8b2011-06-01 20:01:49 +0000550 *ppPage = pWal->apWiData[iPage];
dan13a3cb82010-06-11 19:04:21 +0000551 assert( iPage==0 || *ppPage || rc!=SQLITE_OK );
552 return rc;
553}
554
555/*
drh73b64e42010-05-30 19:55:15 +0000556** Return a pointer to the WalCkptInfo structure in the wal-index.
557*/
558static volatile WalCkptInfo *walCkptInfo(Wal *pWal){
dan4280eb32010-06-12 12:02:35 +0000559 assert( pWal->nWiData>0 && pWal->apWiData[0] );
560 return (volatile WalCkptInfo*)&(pWal->apWiData[0][sizeof(WalIndexHdr)/2]);
561}
562
563/*
564** Return a pointer to the WalIndexHdr structure in the wal-index.
565*/
566static volatile WalIndexHdr *walIndexHdr(Wal *pWal){
567 assert( pWal->nWiData>0 && pWal->apWiData[0] );
568 return (volatile WalIndexHdr*)pWal->apWiData[0];
drh73b64e42010-05-30 19:55:15 +0000569}
570
dan7c246102010-04-12 19:00:29 +0000571/*
danb8fd6c22010-05-24 10:39:36 +0000572** The argument to this macro must be of type u32. On a little-endian
573** architecture, it returns the u32 value that results from interpreting
574** the 4 bytes as a big-endian value. On a big-endian architecture, it
575** returns the value that would be produced by intepreting the 4 bytes
576** of the input value as a little-endian integer.
577*/
578#define BYTESWAP32(x) ( \
579 (((x)&0x000000FF)<<24) + (((x)&0x0000FF00)<<8) \
580 + (((x)&0x00FF0000)>>8) + (((x)&0xFF000000)>>24) \
581)
dan64d039e2010-04-13 19:27:31 +0000582
dan7c246102010-04-12 19:00:29 +0000583/*
drh7e263722010-05-20 21:21:09 +0000584** Generate or extend an 8 byte checksum based on the data in
585** array aByte[] and the initial values of aIn[0] and aIn[1] (or
586** initial values of 0 and 0 if aIn==NULL).
587**
588** The checksum is written back into aOut[] before returning.
589**
590** nByte must be a positive multiple of 8.
dan7c246102010-04-12 19:00:29 +0000591*/
drh7e263722010-05-20 21:21:09 +0000592static void walChecksumBytes(
danb8fd6c22010-05-24 10:39:36 +0000593 int nativeCksum, /* True for native byte-order, false for non-native */
drh7e263722010-05-20 21:21:09 +0000594 u8 *a, /* Content to be checksummed */
595 int nByte, /* Bytes of content in a[]. Must be a multiple of 8. */
596 const u32 *aIn, /* Initial checksum value input */
597 u32 *aOut /* OUT: Final checksum value output */
598){
599 u32 s1, s2;
danb8fd6c22010-05-24 10:39:36 +0000600 u32 *aData = (u32 *)a;
601 u32 *aEnd = (u32 *)&a[nByte];
602
drh7e263722010-05-20 21:21:09 +0000603 if( aIn ){
604 s1 = aIn[0];
605 s2 = aIn[1];
606 }else{
607 s1 = s2 = 0;
608 }
dan7c246102010-04-12 19:00:29 +0000609
drh584c7542010-05-19 18:08:10 +0000610 assert( nByte>=8 );
danb8fd6c22010-05-24 10:39:36 +0000611 assert( (nByte&0x00000007)==0 );
dan7c246102010-04-12 19:00:29 +0000612
danb8fd6c22010-05-24 10:39:36 +0000613 if( nativeCksum ){
614 do {
615 s1 += *aData++ + s2;
616 s2 += *aData++ + s1;
617 }while( aData<aEnd );
618 }else{
619 do {
620 s1 += BYTESWAP32(aData[0]) + s2;
621 s2 += BYTESWAP32(aData[1]) + s1;
622 aData += 2;
623 }while( aData<aEnd );
624 }
625
drh7e263722010-05-20 21:21:09 +0000626 aOut[0] = s1;
627 aOut[1] = s2;
dan7c246102010-04-12 19:00:29 +0000628}
629
dan8c408002010-11-01 17:38:24 +0000630static void walShmBarrier(Wal *pWal){
631 if( pWal->exclusiveMode!=WAL_HEAPMEMORY_MODE ){
632 sqlite3OsShmBarrier(pWal->pDbFd);
633 }
634}
635
dan7c246102010-04-12 19:00:29 +0000636/*
drh7e263722010-05-20 21:21:09 +0000637** Write the header information in pWal->hdr into the wal-index.
638**
639** The checksum on pWal->hdr is updated before it is written.
drh7ed91f22010-04-29 22:34:07 +0000640*/
drh7e263722010-05-20 21:21:09 +0000641static void walIndexWriteHdr(Wal *pWal){
dan4280eb32010-06-12 12:02:35 +0000642 volatile WalIndexHdr *aHdr = walIndexHdr(pWal);
643 const int nCksum = offsetof(WalIndexHdr, aCksum);
drh73b64e42010-05-30 19:55:15 +0000644
645 assert( pWal->writeLock );
drh4b82c382010-05-31 18:24:19 +0000646 pWal->hdr.isInit = 1;
dan10f5a502010-06-23 15:55:43 +0000647 pWal->hdr.iVersion = WALINDEX_MAX_VERSION;
dan4280eb32010-06-12 12:02:35 +0000648 walChecksumBytes(1, (u8*)&pWal->hdr, nCksum, 0, pWal->hdr.aCksum);
649 memcpy((void *)&aHdr[1], (void *)&pWal->hdr, sizeof(WalIndexHdr));
dan8c408002010-11-01 17:38:24 +0000650 walShmBarrier(pWal);
dan4280eb32010-06-12 12:02:35 +0000651 memcpy((void *)&aHdr[0], (void *)&pWal->hdr, sizeof(WalIndexHdr));
dan7c246102010-04-12 19:00:29 +0000652}
653
654/*
655** This function encodes a single frame header and writes it to a buffer
drh7ed91f22010-04-29 22:34:07 +0000656** supplied by the caller. A frame-header is made up of a series of
dan7c246102010-04-12 19:00:29 +0000657** 4-byte big-endian integers, as follows:
658**
drh23ea97b2010-05-20 16:45:58 +0000659** 0: Page number.
660** 4: For commit records, the size of the database image in pages
661** after the commit. For all other records, zero.
drh7e263722010-05-20 21:21:09 +0000662** 8: Salt-1 (copied from the wal-header)
663** 12: Salt-2 (copied from the wal-header)
drh23ea97b2010-05-20 16:45:58 +0000664** 16: Checksum-1.
665** 20: Checksum-2.
dan7c246102010-04-12 19:00:29 +0000666*/
drh7ed91f22010-04-29 22:34:07 +0000667static void walEncodeFrame(
drh23ea97b2010-05-20 16:45:58 +0000668 Wal *pWal, /* The write-ahead log */
dan7c246102010-04-12 19:00:29 +0000669 u32 iPage, /* Database page number for frame */
670 u32 nTruncate, /* New db size (or 0 for non-commit frames) */
drh7e263722010-05-20 21:21:09 +0000671 u8 *aData, /* Pointer to page data */
dan7c246102010-04-12 19:00:29 +0000672 u8 *aFrame /* OUT: Write encoded frame here */
673){
danb8fd6c22010-05-24 10:39:36 +0000674 int nativeCksum; /* True for native byte-order checksums */
dan71d89912010-05-24 13:57:42 +0000675 u32 *aCksum = pWal->hdr.aFrameCksum;
drh23ea97b2010-05-20 16:45:58 +0000676 assert( WAL_FRAME_HDRSIZE==24 );
dan97a31352010-04-16 13:59:31 +0000677 sqlite3Put4byte(&aFrame[0], iPage);
678 sqlite3Put4byte(&aFrame[4], nTruncate);
drh7e263722010-05-20 21:21:09 +0000679 memcpy(&aFrame[8], pWal->hdr.aSalt, 8);
dan7c246102010-04-12 19:00:29 +0000680
danb8fd6c22010-05-24 10:39:36 +0000681 nativeCksum = (pWal->hdr.bigEndCksum==SQLITE_BIGENDIAN);
dan71d89912010-05-24 13:57:42 +0000682 walChecksumBytes(nativeCksum, aFrame, 8, aCksum, aCksum);
danb8fd6c22010-05-24 10:39:36 +0000683 walChecksumBytes(nativeCksum, aData, pWal->szPage, aCksum, aCksum);
dan7c246102010-04-12 19:00:29 +0000684
drh23ea97b2010-05-20 16:45:58 +0000685 sqlite3Put4byte(&aFrame[16], aCksum[0]);
686 sqlite3Put4byte(&aFrame[20], aCksum[1]);
dan7c246102010-04-12 19:00:29 +0000687}
688
689/*
drh7e263722010-05-20 21:21:09 +0000690** Check to see if the frame with header in aFrame[] and content
691** in aData[] is valid. If it is a valid frame, fill *piPage and
692** *pnTruncate and return true. Return if the frame is not valid.
dan7c246102010-04-12 19:00:29 +0000693*/
drh7ed91f22010-04-29 22:34:07 +0000694static int walDecodeFrame(
drh23ea97b2010-05-20 16:45:58 +0000695 Wal *pWal, /* The write-ahead log */
dan7c246102010-04-12 19:00:29 +0000696 u32 *piPage, /* OUT: Database page number for frame */
697 u32 *pnTruncate, /* OUT: New db size (or 0 if not commit) */
dan7c246102010-04-12 19:00:29 +0000698 u8 *aData, /* Pointer to page data (for checksum) */
699 u8 *aFrame /* Frame data */
700){
danb8fd6c22010-05-24 10:39:36 +0000701 int nativeCksum; /* True for native byte-order checksums */
dan71d89912010-05-24 13:57:42 +0000702 u32 *aCksum = pWal->hdr.aFrameCksum;
drhc8179152010-05-24 13:28:36 +0000703 u32 pgno; /* Page number of the frame */
drh23ea97b2010-05-20 16:45:58 +0000704 assert( WAL_FRAME_HDRSIZE==24 );
705
drh7e263722010-05-20 21:21:09 +0000706 /* A frame is only valid if the salt values in the frame-header
707 ** match the salt values in the wal-header.
708 */
709 if( memcmp(&pWal->hdr.aSalt, &aFrame[8], 8)!=0 ){
drh23ea97b2010-05-20 16:45:58 +0000710 return 0;
711 }
dan4a4b01d2010-04-16 11:30:18 +0000712
drhc8179152010-05-24 13:28:36 +0000713 /* A frame is only valid if the page number is creater than zero.
714 */
715 pgno = sqlite3Get4byte(&aFrame[0]);
716 if( pgno==0 ){
717 return 0;
718 }
719
drh519426a2010-07-09 03:19:07 +0000720 /* A frame is only valid if a checksum of the WAL header,
721 ** all prior frams, the first 16 bytes of this frame-header,
722 ** and the frame-data matches the checksum in the last 8
723 ** bytes of this frame-header.
drh7e263722010-05-20 21:21:09 +0000724 */
danb8fd6c22010-05-24 10:39:36 +0000725 nativeCksum = (pWal->hdr.bigEndCksum==SQLITE_BIGENDIAN);
dan71d89912010-05-24 13:57:42 +0000726 walChecksumBytes(nativeCksum, aFrame, 8, aCksum, aCksum);
danb8fd6c22010-05-24 10:39:36 +0000727 walChecksumBytes(nativeCksum, aData, pWal->szPage, aCksum, aCksum);
drh23ea97b2010-05-20 16:45:58 +0000728 if( aCksum[0]!=sqlite3Get4byte(&aFrame[16])
729 || aCksum[1]!=sqlite3Get4byte(&aFrame[20])
dan7c246102010-04-12 19:00:29 +0000730 ){
731 /* Checksum failed. */
732 return 0;
733 }
734
drh7e263722010-05-20 21:21:09 +0000735 /* If we reach this point, the frame is valid. Return the page number
736 ** and the new database size.
737 */
drhc8179152010-05-24 13:28:36 +0000738 *piPage = pgno;
dan97a31352010-04-16 13:59:31 +0000739 *pnTruncate = sqlite3Get4byte(&aFrame[4]);
dan7c246102010-04-12 19:00:29 +0000740 return 1;
741}
742
dan7c246102010-04-12 19:00:29 +0000743
drhc74c3332010-05-31 12:15:19 +0000744#if defined(SQLITE_TEST) && defined(SQLITE_DEBUG)
745/*
drh181e0912010-06-01 01:08:08 +0000746** Names of locks. This routine is used to provide debugging output and is not
747** a part of an ordinary build.
drhc74c3332010-05-31 12:15:19 +0000748*/
749static const char *walLockName(int lockIdx){
750 if( lockIdx==WAL_WRITE_LOCK ){
751 return "WRITE-LOCK";
752 }else if( lockIdx==WAL_CKPT_LOCK ){
753 return "CKPT-LOCK";
754 }else if( lockIdx==WAL_RECOVER_LOCK ){
755 return "RECOVER-LOCK";
756 }else{
757 static char zName[15];
758 sqlite3_snprintf(sizeof(zName), zName, "READ-LOCK[%d]",
759 lockIdx-WAL_READ_LOCK(0));
760 return zName;
761 }
762}
763#endif /*defined(SQLITE_TEST) || defined(SQLITE_DEBUG) */
764
765
dan7c246102010-04-12 19:00:29 +0000766/*
drh181e0912010-06-01 01:08:08 +0000767** Set or release locks on the WAL. Locks are either shared or exclusive.
768** A lock cannot be moved directly between shared and exclusive - it must go
769** through the unlocked state first.
drh73b64e42010-05-30 19:55:15 +0000770**
771** In locking_mode=EXCLUSIVE, all of these routines become no-ops.
772*/
773static int walLockShared(Wal *pWal, int lockIdx){
drhc74c3332010-05-31 12:15:19 +0000774 int rc;
drh73b64e42010-05-30 19:55:15 +0000775 if( pWal->exclusiveMode ) return SQLITE_OK;
drhc74c3332010-05-31 12:15:19 +0000776 rc = sqlite3OsShmLock(pWal->pDbFd, lockIdx, 1,
777 SQLITE_SHM_LOCK | SQLITE_SHM_SHARED);
778 WALTRACE(("WAL%p: acquire SHARED-%s %s\n", pWal,
779 walLockName(lockIdx), rc ? "failed" : "ok"));
shaneh5eba1f62010-07-02 17:05:03 +0000780 VVA_ONLY( pWal->lockError = (u8)(rc!=SQLITE_OK && rc!=SQLITE_BUSY); )
drhc74c3332010-05-31 12:15:19 +0000781 return rc;
drh73b64e42010-05-30 19:55:15 +0000782}
783static void walUnlockShared(Wal *pWal, int lockIdx){
784 if( pWal->exclusiveMode ) return;
785 (void)sqlite3OsShmLock(pWal->pDbFd, lockIdx, 1,
786 SQLITE_SHM_UNLOCK | SQLITE_SHM_SHARED);
drhc74c3332010-05-31 12:15:19 +0000787 WALTRACE(("WAL%p: release SHARED-%s\n", pWal, walLockName(lockIdx)));
drh73b64e42010-05-30 19:55:15 +0000788}
789static int walLockExclusive(Wal *pWal, int lockIdx, int n){
drhc74c3332010-05-31 12:15:19 +0000790 int rc;
drh73b64e42010-05-30 19:55:15 +0000791 if( pWal->exclusiveMode ) return SQLITE_OK;
drhc74c3332010-05-31 12:15:19 +0000792 rc = sqlite3OsShmLock(pWal->pDbFd, lockIdx, n,
793 SQLITE_SHM_LOCK | SQLITE_SHM_EXCLUSIVE);
794 WALTRACE(("WAL%p: acquire EXCLUSIVE-%s cnt=%d %s\n", pWal,
795 walLockName(lockIdx), n, rc ? "failed" : "ok"));
shaneh5eba1f62010-07-02 17:05:03 +0000796 VVA_ONLY( pWal->lockError = (u8)(rc!=SQLITE_OK && rc!=SQLITE_BUSY); )
drhc74c3332010-05-31 12:15:19 +0000797 return rc;
drh73b64e42010-05-30 19:55:15 +0000798}
799static void walUnlockExclusive(Wal *pWal, int lockIdx, int n){
800 if( pWal->exclusiveMode ) return;
801 (void)sqlite3OsShmLock(pWal->pDbFd, lockIdx, n,
802 SQLITE_SHM_UNLOCK | SQLITE_SHM_EXCLUSIVE);
drhc74c3332010-05-31 12:15:19 +0000803 WALTRACE(("WAL%p: release EXCLUSIVE-%s cnt=%d\n", pWal,
804 walLockName(lockIdx), n));
drh73b64e42010-05-30 19:55:15 +0000805}
806
807/*
drh29d4dbe2010-05-18 23:29:52 +0000808** Compute a hash on a page number. The resulting hash value must land
drh181e0912010-06-01 01:08:08 +0000809** between 0 and (HASHTABLE_NSLOT-1). The walHashNext() function advances
810** the hash to the next value in the event of a collision.
drh29d4dbe2010-05-18 23:29:52 +0000811*/
812static int walHash(u32 iPage){
813 assert( iPage>0 );
814 assert( (HASHTABLE_NSLOT & (HASHTABLE_NSLOT-1))==0 );
815 return (iPage*HASHTABLE_HASH_1) & (HASHTABLE_NSLOT-1);
816}
817static int walNextHash(int iPriorHash){
818 return (iPriorHash+1)&(HASHTABLE_NSLOT-1);
danbb23aff2010-05-10 14:46:09 +0000819}
820
dan4280eb32010-06-12 12:02:35 +0000821/*
822** Return pointers to the hash table and page number array stored on
823** page iHash of the wal-index. The wal-index is broken into 32KB pages
824** numbered starting from 0.
825**
826** Set output variable *paHash to point to the start of the hash table
827** in the wal-index file. Set *piZero to one less than the frame
828** number of the first frame indexed by this hash table. If a
829** slot in the hash table is set to N, it refers to frame number
830** (*piZero+N) in the log.
831**
dand60bf112010-06-14 11:18:50 +0000832** Finally, set *paPgno so that *paPgno[1] is the page number of the
833** first frame indexed by the hash table, frame (*piZero+1).
dan4280eb32010-06-12 12:02:35 +0000834*/
835static int walHashGet(
dan13a3cb82010-06-11 19:04:21 +0000836 Wal *pWal, /* WAL handle */
837 int iHash, /* Find the iHash'th table */
dan067f3162010-06-14 10:30:12 +0000838 volatile ht_slot **paHash, /* OUT: Pointer to hash index */
dan13a3cb82010-06-11 19:04:21 +0000839 volatile u32 **paPgno, /* OUT: Pointer to page number array */
840 u32 *piZero /* OUT: Frame associated with *paPgno[0] */
841){
dan4280eb32010-06-12 12:02:35 +0000842 int rc; /* Return code */
dan13a3cb82010-06-11 19:04:21 +0000843 volatile u32 *aPgno;
dan13a3cb82010-06-11 19:04:21 +0000844
dan4280eb32010-06-12 12:02:35 +0000845 rc = walIndexPage(pWal, iHash, &aPgno);
846 assert( rc==SQLITE_OK || iHash>0 );
dan13a3cb82010-06-11 19:04:21 +0000847
dan4280eb32010-06-12 12:02:35 +0000848 if( rc==SQLITE_OK ){
849 u32 iZero;
dan067f3162010-06-14 10:30:12 +0000850 volatile ht_slot *aHash;
dan4280eb32010-06-12 12:02:35 +0000851
dan067f3162010-06-14 10:30:12 +0000852 aHash = (volatile ht_slot *)&aPgno[HASHTABLE_NPAGE];
dan4280eb32010-06-12 12:02:35 +0000853 if( iHash==0 ){
dand60bf112010-06-14 11:18:50 +0000854 aPgno = &aPgno[WALINDEX_HDR_SIZE/sizeof(u32)];
dan4280eb32010-06-12 12:02:35 +0000855 iZero = 0;
856 }else{
857 iZero = HASHTABLE_NPAGE_ONE + (iHash-1)*HASHTABLE_NPAGE;
dan4280eb32010-06-12 12:02:35 +0000858 }
859
dand60bf112010-06-14 11:18:50 +0000860 *paPgno = &aPgno[-1];
dan4280eb32010-06-12 12:02:35 +0000861 *paHash = aHash;
862 *piZero = iZero;
dan13a3cb82010-06-11 19:04:21 +0000863 }
dan4280eb32010-06-12 12:02:35 +0000864 return rc;
dan13a3cb82010-06-11 19:04:21 +0000865}
866
dan4280eb32010-06-12 12:02:35 +0000867/*
868** Return the number of the wal-index page that contains the hash-table
869** and page-number array that contain entries corresponding to WAL frame
870** iFrame. The wal-index is broken up into 32KB pages. Wal-index pages
871** are numbered starting from 0.
872*/
dan13a3cb82010-06-11 19:04:21 +0000873static int walFramePage(u32 iFrame){
874 int iHash = (iFrame+HASHTABLE_NPAGE-HASHTABLE_NPAGE_ONE-1) / HASHTABLE_NPAGE;
875 assert( (iHash==0 || iFrame>HASHTABLE_NPAGE_ONE)
876 && (iHash>=1 || iFrame<=HASHTABLE_NPAGE_ONE)
877 && (iHash<=1 || iFrame>(HASHTABLE_NPAGE_ONE+HASHTABLE_NPAGE))
878 && (iHash>=2 || iFrame<=HASHTABLE_NPAGE_ONE+HASHTABLE_NPAGE)
879 && (iHash<=2 || iFrame>(HASHTABLE_NPAGE_ONE+2*HASHTABLE_NPAGE))
880 );
881 return iHash;
882}
883
884/*
885** Return the page number associated with frame iFrame in this WAL.
886*/
887static u32 walFramePgno(Wal *pWal, u32 iFrame){
888 int iHash = walFramePage(iFrame);
889 if( iHash==0 ){
890 return pWal->apWiData[0][WALINDEX_HDR_SIZE/sizeof(u32) + iFrame - 1];
891 }
892 return pWal->apWiData[iHash][(iFrame-1-HASHTABLE_NPAGE_ONE)%HASHTABLE_NPAGE];
893}
danbb23aff2010-05-10 14:46:09 +0000894
danca6b5ba2010-05-25 10:50:56 +0000895/*
896** Remove entries from the hash table that point to WAL slots greater
897** than pWal->hdr.mxFrame.
898**
899** This function is called whenever pWal->hdr.mxFrame is decreased due
900** to a rollback or savepoint.
901**
drh181e0912010-06-01 01:08:08 +0000902** At most only the hash table containing pWal->hdr.mxFrame needs to be
903** updated. Any later hash tables will be automatically cleared when
904** pWal->hdr.mxFrame advances to the point where those hash tables are
905** actually needed.
danca6b5ba2010-05-25 10:50:56 +0000906*/
907static void walCleanupHash(Wal *pWal){
drhff828942010-06-26 21:34:06 +0000908 volatile ht_slot *aHash = 0; /* Pointer to hash table to clear */
909 volatile u32 *aPgno = 0; /* Page number array for hash table */
910 u32 iZero = 0; /* frame == (aHash[x]+iZero) */
dan067f3162010-06-14 10:30:12 +0000911 int iLimit = 0; /* Zero values greater than this */
912 int nByte; /* Number of bytes to zero in aPgno[] */
913 int i; /* Used to iterate through aHash[] */
danca6b5ba2010-05-25 10:50:56 +0000914
drh73b64e42010-05-30 19:55:15 +0000915 assert( pWal->writeLock );
drhffca4302010-06-15 11:21:54 +0000916 testcase( pWal->hdr.mxFrame==HASHTABLE_NPAGE_ONE-1 );
917 testcase( pWal->hdr.mxFrame==HASHTABLE_NPAGE_ONE );
918 testcase( pWal->hdr.mxFrame==HASHTABLE_NPAGE_ONE+1 );
drh9c156472010-06-01 12:58:41 +0000919
dan4280eb32010-06-12 12:02:35 +0000920 if( pWal->hdr.mxFrame==0 ) return;
921
922 /* Obtain pointers to the hash-table and page-number array containing
923 ** the entry that corresponds to frame pWal->hdr.mxFrame. It is guaranteed
924 ** that the page said hash-table and array reside on is already mapped.
925 */
926 assert( pWal->nWiData>walFramePage(pWal->hdr.mxFrame) );
927 assert( pWal->apWiData[walFramePage(pWal->hdr.mxFrame)] );
928 walHashGet(pWal, walFramePage(pWal->hdr.mxFrame), &aHash, &aPgno, &iZero);
929
930 /* Zero all hash-table entries that correspond to frame numbers greater
931 ** than pWal->hdr.mxFrame.
932 */
933 iLimit = pWal->hdr.mxFrame - iZero;
934 assert( iLimit>0 );
935 for(i=0; i<HASHTABLE_NSLOT; i++){
936 if( aHash[i]>iLimit ){
937 aHash[i] = 0;
danca6b5ba2010-05-25 10:50:56 +0000938 }
danca6b5ba2010-05-25 10:50:56 +0000939 }
dan4280eb32010-06-12 12:02:35 +0000940
941 /* Zero the entries in the aPgno array that correspond to frames with
942 ** frame numbers greater than pWal->hdr.mxFrame.
943 */
shaneh5eba1f62010-07-02 17:05:03 +0000944 nByte = (int)((char *)aHash - (char *)&aPgno[iLimit+1]);
dand60bf112010-06-14 11:18:50 +0000945 memset((void *)&aPgno[iLimit+1], 0, nByte);
danca6b5ba2010-05-25 10:50:56 +0000946
947#ifdef SQLITE_ENABLE_EXPENSIVE_ASSERT
948 /* Verify that the every entry in the mapping region is still reachable
949 ** via the hash table even after the cleanup.
950 */
drhf77bbd92010-06-01 13:17:44 +0000951 if( iLimit ){
danca6b5ba2010-05-25 10:50:56 +0000952 int i; /* Loop counter */
953 int iKey; /* Hash key */
954 for(i=1; i<=iLimit; i++){
dand60bf112010-06-14 11:18:50 +0000955 for(iKey=walHash(aPgno[i]); aHash[iKey]; iKey=walNextHash(iKey)){
danca6b5ba2010-05-25 10:50:56 +0000956 if( aHash[iKey]==i ) break;
957 }
958 assert( aHash[iKey]==i );
959 }
960 }
961#endif /* SQLITE_ENABLE_EXPENSIVE_ASSERT */
962}
963
danbb23aff2010-05-10 14:46:09 +0000964
drh7ed91f22010-04-29 22:34:07 +0000965/*
drh29d4dbe2010-05-18 23:29:52 +0000966** Set an entry in the wal-index that will map database page number
967** pPage into WAL frame iFrame.
dan7c246102010-04-12 19:00:29 +0000968*/
drh7ed91f22010-04-29 22:34:07 +0000969static int walIndexAppend(Wal *pWal, u32 iFrame, u32 iPage){
dan4280eb32010-06-12 12:02:35 +0000970 int rc; /* Return code */
drhff828942010-06-26 21:34:06 +0000971 u32 iZero = 0; /* One less than frame number of aPgno[1] */
972 volatile u32 *aPgno = 0; /* Page number array */
973 volatile ht_slot *aHash = 0; /* Hash table */
dance4f05f2010-04-22 19:14:13 +0000974
dan4280eb32010-06-12 12:02:35 +0000975 rc = walHashGet(pWal, walFramePage(iFrame), &aHash, &aPgno, &iZero);
976
977 /* Assuming the wal-index file was successfully mapped, populate the
978 ** page number array and hash table entry.
dan7c246102010-04-12 19:00:29 +0000979 */
danbb23aff2010-05-10 14:46:09 +0000980 if( rc==SQLITE_OK ){
981 int iKey; /* Hash table key */
dan4280eb32010-06-12 12:02:35 +0000982 int idx; /* Value to write to hash-table slot */
drh519426a2010-07-09 03:19:07 +0000983 int nCollide; /* Number of hash collisions */
dan7c246102010-04-12 19:00:29 +0000984
danbb23aff2010-05-10 14:46:09 +0000985 idx = iFrame - iZero;
dan4280eb32010-06-12 12:02:35 +0000986 assert( idx <= HASHTABLE_NSLOT/2 + 1 );
987
988 /* If this is the first entry to be added to this hash-table, zero the
989 ** entire hash table and aPgno[] array before proceding.
990 */
danca6b5ba2010-05-25 10:50:56 +0000991 if( idx==1 ){
shaneh5eba1f62010-07-02 17:05:03 +0000992 int nByte = (int)((u8 *)&aHash[HASHTABLE_NSLOT] - (u8 *)&aPgno[1]);
dand60bf112010-06-14 11:18:50 +0000993 memset((void*)&aPgno[1], 0, nByte);
danca6b5ba2010-05-25 10:50:56 +0000994 }
danca6b5ba2010-05-25 10:50:56 +0000995
dan4280eb32010-06-12 12:02:35 +0000996 /* If the entry in aPgno[] is already set, then the previous writer
997 ** must have exited unexpectedly in the middle of a transaction (after
998 ** writing one or more dirty pages to the WAL to free up memory).
999 ** Remove the remnants of that writers uncommitted transaction from
1000 ** the hash-table before writing any new entries.
1001 */
dand60bf112010-06-14 11:18:50 +00001002 if( aPgno[idx] ){
danca6b5ba2010-05-25 10:50:56 +00001003 walCleanupHash(pWal);
dand60bf112010-06-14 11:18:50 +00001004 assert( !aPgno[idx] );
danca6b5ba2010-05-25 10:50:56 +00001005 }
dan4280eb32010-06-12 12:02:35 +00001006
1007 /* Write the aPgno[] array entry and the hash-table slot. */
drh519426a2010-07-09 03:19:07 +00001008 nCollide = idx;
dan6f150142010-05-21 15:31:56 +00001009 for(iKey=walHash(iPage); aHash[iKey]; iKey=walNextHash(iKey)){
drh519426a2010-07-09 03:19:07 +00001010 if( (nCollide--)==0 ) return SQLITE_CORRUPT_BKPT;
drh29d4dbe2010-05-18 23:29:52 +00001011 }
dand60bf112010-06-14 11:18:50 +00001012 aPgno[idx] = iPage;
shaneh5eba1f62010-07-02 17:05:03 +00001013 aHash[iKey] = (ht_slot)idx;
drh4fa95bf2010-05-22 00:55:39 +00001014
1015#ifdef SQLITE_ENABLE_EXPENSIVE_ASSERT
1016 /* Verify that the number of entries in the hash table exactly equals
1017 ** the number of entries in the mapping region.
1018 */
1019 {
1020 int i; /* Loop counter */
1021 int nEntry = 0; /* Number of entries in the hash table */
1022 for(i=0; i<HASHTABLE_NSLOT; i++){ if( aHash[i] ) nEntry++; }
1023 assert( nEntry==idx );
1024 }
1025
1026 /* Verify that the every entry in the mapping region is reachable
1027 ** via the hash table. This turns out to be a really, really expensive
1028 ** thing to check, so only do this occasionally - not on every
1029 ** iteration.
1030 */
1031 if( (idx&0x3ff)==0 ){
1032 int i; /* Loop counter */
1033 for(i=1; i<=idx; i++){
dand60bf112010-06-14 11:18:50 +00001034 for(iKey=walHash(aPgno[i]); aHash[iKey]; iKey=walNextHash(iKey)){
drh4fa95bf2010-05-22 00:55:39 +00001035 if( aHash[iKey]==i ) break;
1036 }
1037 assert( aHash[iKey]==i );
1038 }
1039 }
1040#endif /* SQLITE_ENABLE_EXPENSIVE_ASSERT */
dan7c246102010-04-12 19:00:29 +00001041 }
dan31f98fc2010-04-27 05:42:32 +00001042
drh4fa95bf2010-05-22 00:55:39 +00001043
danbb23aff2010-05-10 14:46:09 +00001044 return rc;
dan7c246102010-04-12 19:00:29 +00001045}
1046
1047
1048/*
drh7ed91f22010-04-29 22:34:07 +00001049** Recover the wal-index by reading the write-ahead log file.
drh73b64e42010-05-30 19:55:15 +00001050**
1051** This routine first tries to establish an exclusive lock on the
1052** wal-index to prevent other threads/processes from doing anything
1053** with the WAL or wal-index while recovery is running. The
1054** WAL_RECOVER_LOCK is also held so that other threads will know
1055** that this thread is running recovery. If unable to establish
1056** the necessary locks, this routine returns SQLITE_BUSY.
dan7c246102010-04-12 19:00:29 +00001057*/
drh7ed91f22010-04-29 22:34:07 +00001058static int walIndexRecover(Wal *pWal){
dan7c246102010-04-12 19:00:29 +00001059 int rc; /* Return Code */
1060 i64 nSize; /* Size of log file */
dan71d89912010-05-24 13:57:42 +00001061 u32 aFrameCksum[2] = {0, 0};
dand0aa3422010-05-31 16:41:53 +00001062 int iLock; /* Lock offset to lock for checkpoint */
1063 int nLock; /* Number of locks to hold */
dan7c246102010-04-12 19:00:29 +00001064
dand0aa3422010-05-31 16:41:53 +00001065 /* Obtain an exclusive lock on all byte in the locking range not already
1066 ** locked by the caller. The caller is guaranteed to have locked the
1067 ** WAL_WRITE_LOCK byte, and may have also locked the WAL_CKPT_LOCK byte.
1068 ** If successful, the same bytes that are locked here are unlocked before
1069 ** this function returns.
1070 */
1071 assert( pWal->ckptLock==1 || pWal->ckptLock==0 );
1072 assert( WAL_ALL_BUT_WRITE==WAL_WRITE_LOCK+1 );
1073 assert( WAL_CKPT_LOCK==WAL_ALL_BUT_WRITE );
1074 assert( pWal->writeLock );
1075 iLock = WAL_ALL_BUT_WRITE + pWal->ckptLock;
1076 nLock = SQLITE_SHM_NLOCK - iLock;
1077 rc = walLockExclusive(pWal, iLock, nLock);
drh73b64e42010-05-30 19:55:15 +00001078 if( rc ){
1079 return rc;
1080 }
drhc74c3332010-05-31 12:15:19 +00001081 WALTRACE(("WAL%p: recovery begin...\n", pWal));
drh73b64e42010-05-30 19:55:15 +00001082
dan71d89912010-05-24 13:57:42 +00001083 memset(&pWal->hdr, 0, sizeof(WalIndexHdr));
dan7c246102010-04-12 19:00:29 +00001084
drhd9e5c4f2010-05-12 18:01:39 +00001085 rc = sqlite3OsFileSize(pWal->pWalFd, &nSize);
dan7c246102010-04-12 19:00:29 +00001086 if( rc!=SQLITE_OK ){
drh73b64e42010-05-30 19:55:15 +00001087 goto recovery_error;
dan7c246102010-04-12 19:00:29 +00001088 }
1089
danb8fd6c22010-05-24 10:39:36 +00001090 if( nSize>WAL_HDRSIZE ){
1091 u8 aBuf[WAL_HDRSIZE]; /* Buffer to load WAL header into */
dan7c246102010-04-12 19:00:29 +00001092 u8 *aFrame = 0; /* Malloc'd buffer to load entire frame */
drh584c7542010-05-19 18:08:10 +00001093 int szFrame; /* Number of bytes in buffer aFrame[] */
dan7c246102010-04-12 19:00:29 +00001094 u8 *aData; /* Pointer to data part of aFrame buffer */
1095 int iFrame; /* Index of last frame read */
1096 i64 iOffset; /* Next offset to read from log file */
drh6e810962010-05-19 17:49:50 +00001097 int szPage; /* Page size according to the log */
danb8fd6c22010-05-24 10:39:36 +00001098 u32 magic; /* Magic value read from WAL header */
dan10f5a502010-06-23 15:55:43 +00001099 u32 version; /* Magic value read from WAL header */
dan7c246102010-04-12 19:00:29 +00001100
danb8fd6c22010-05-24 10:39:36 +00001101 /* Read in the WAL header. */
drhd9e5c4f2010-05-12 18:01:39 +00001102 rc = sqlite3OsRead(pWal->pWalFd, aBuf, WAL_HDRSIZE, 0);
dan7c246102010-04-12 19:00:29 +00001103 if( rc!=SQLITE_OK ){
drh73b64e42010-05-30 19:55:15 +00001104 goto recovery_error;
dan7c246102010-04-12 19:00:29 +00001105 }
1106
1107 /* If the database page size is not a power of two, or is greater than
danb8fd6c22010-05-24 10:39:36 +00001108 ** SQLITE_MAX_PAGE_SIZE, conclude that the WAL file contains no valid
1109 ** data. Similarly, if the 'magic' value is invalid, ignore the whole
1110 ** WAL file.
dan7c246102010-04-12 19:00:29 +00001111 */
danb8fd6c22010-05-24 10:39:36 +00001112 magic = sqlite3Get4byte(&aBuf[0]);
drh23ea97b2010-05-20 16:45:58 +00001113 szPage = sqlite3Get4byte(&aBuf[8]);
danb8fd6c22010-05-24 10:39:36 +00001114 if( (magic&0xFFFFFFFE)!=WAL_MAGIC
1115 || szPage&(szPage-1)
1116 || szPage>SQLITE_MAX_PAGE_SIZE
1117 || szPage<512
1118 ){
dan7c246102010-04-12 19:00:29 +00001119 goto finished;
1120 }
shaneh5eba1f62010-07-02 17:05:03 +00001121 pWal->hdr.bigEndCksum = (u8)(magic&0x00000001);
drhb2eced52010-08-12 02:41:12 +00001122 pWal->szPage = szPage;
drh23ea97b2010-05-20 16:45:58 +00001123 pWal->nCkpt = sqlite3Get4byte(&aBuf[12]);
drh7e263722010-05-20 21:21:09 +00001124 memcpy(&pWal->hdr.aSalt, &aBuf[16], 8);
drhcd285082010-06-23 22:00:35 +00001125
1126 /* Verify that the WAL header checksum is correct */
dan71d89912010-05-24 13:57:42 +00001127 walChecksumBytes(pWal->hdr.bigEndCksum==SQLITE_BIGENDIAN,
dan10f5a502010-06-23 15:55:43 +00001128 aBuf, WAL_HDRSIZE-2*4, 0, pWal->hdr.aFrameCksum
dan71d89912010-05-24 13:57:42 +00001129 );
dan10f5a502010-06-23 15:55:43 +00001130 if( pWal->hdr.aFrameCksum[0]!=sqlite3Get4byte(&aBuf[24])
1131 || pWal->hdr.aFrameCksum[1]!=sqlite3Get4byte(&aBuf[28])
1132 ){
1133 goto finished;
1134 }
1135
drhcd285082010-06-23 22:00:35 +00001136 /* Verify that the version number on the WAL format is one that
1137 ** are able to understand */
dan10f5a502010-06-23 15:55:43 +00001138 version = sqlite3Get4byte(&aBuf[4]);
1139 if( version!=WAL_MAX_VERSION ){
1140 rc = SQLITE_CANTOPEN_BKPT;
1141 goto finished;
1142 }
1143
dan7c246102010-04-12 19:00:29 +00001144 /* Malloc a buffer to read frames into. */
drh584c7542010-05-19 18:08:10 +00001145 szFrame = szPage + WAL_FRAME_HDRSIZE;
1146 aFrame = (u8 *)sqlite3_malloc(szFrame);
dan7c246102010-04-12 19:00:29 +00001147 if( !aFrame ){
drh73b64e42010-05-30 19:55:15 +00001148 rc = SQLITE_NOMEM;
1149 goto recovery_error;
dan7c246102010-04-12 19:00:29 +00001150 }
drh7ed91f22010-04-29 22:34:07 +00001151 aData = &aFrame[WAL_FRAME_HDRSIZE];
dan7c246102010-04-12 19:00:29 +00001152
1153 /* Read all frames from the log file. */
1154 iFrame = 0;
drh584c7542010-05-19 18:08:10 +00001155 for(iOffset=WAL_HDRSIZE; (iOffset+szFrame)<=nSize; iOffset+=szFrame){
dan7c246102010-04-12 19:00:29 +00001156 u32 pgno; /* Database page number for frame */
1157 u32 nTruncate; /* dbsize field from frame header */
1158 int isValid; /* True if this frame is valid */
1159
1160 /* Read and decode the next log frame. */
drh584c7542010-05-19 18:08:10 +00001161 rc = sqlite3OsRead(pWal->pWalFd, aFrame, szFrame, iOffset);
dan7c246102010-04-12 19:00:29 +00001162 if( rc!=SQLITE_OK ) break;
drh7e263722010-05-20 21:21:09 +00001163 isValid = walDecodeFrame(pWal, &pgno, &nTruncate, aData, aFrame);
dan7c246102010-04-12 19:00:29 +00001164 if( !isValid ) break;
danc7991bd2010-05-05 19:04:59 +00001165 rc = walIndexAppend(pWal, ++iFrame, pgno);
1166 if( rc!=SQLITE_OK ) break;
dan7c246102010-04-12 19:00:29 +00001167
1168 /* If nTruncate is non-zero, this is a commit record. */
1169 if( nTruncate ){
dan71d89912010-05-24 13:57:42 +00001170 pWal->hdr.mxFrame = iFrame;
1171 pWal->hdr.nPage = nTruncate;
shaneh1df2db72010-08-18 02:28:48 +00001172 pWal->hdr.szPage = (u16)((szPage&0xff00) | (szPage>>16));
drh9b78f792010-08-14 21:21:24 +00001173 testcase( szPage<=32768 );
1174 testcase( szPage>=65536 );
dan71d89912010-05-24 13:57:42 +00001175 aFrameCksum[0] = pWal->hdr.aFrameCksum[0];
1176 aFrameCksum[1] = pWal->hdr.aFrameCksum[1];
dan7c246102010-04-12 19:00:29 +00001177 }
1178 }
1179
1180 sqlite3_free(aFrame);
dan7c246102010-04-12 19:00:29 +00001181 }
1182
1183finished:
dan576bc322010-05-06 18:04:50 +00001184 if( rc==SQLITE_OK ){
drhdb7f6472010-06-09 14:45:12 +00001185 volatile WalCkptInfo *pInfo;
1186 int i;
dan71d89912010-05-24 13:57:42 +00001187 pWal->hdr.aFrameCksum[0] = aFrameCksum[0];
1188 pWal->hdr.aFrameCksum[1] = aFrameCksum[1];
drh7e263722010-05-20 21:21:09 +00001189 walIndexWriteHdr(pWal);
dan3dee6da2010-05-31 16:17:54 +00001190
drhdb7f6472010-06-09 14:45:12 +00001191 /* Reset the checkpoint-header. This is safe because this thread is
dan3dee6da2010-05-31 16:17:54 +00001192 ** currently holding locks that exclude all other readers, writers and
1193 ** checkpointers.
1194 */
drhdb7f6472010-06-09 14:45:12 +00001195 pInfo = walCkptInfo(pWal);
1196 pInfo->nBackfill = 0;
1197 pInfo->aReadMark[0] = 0;
1198 for(i=1; i<WAL_NREADER; i++) pInfo->aReadMark[i] = READMARK_NOT_USED;
daneb8763d2010-08-17 14:52:22 +00001199
1200 /* If more than one frame was recovered from the log file, report an
1201 ** event via sqlite3_log(). This is to help with identifying performance
1202 ** problems caused by applications routinely shutting down without
1203 ** checkpointing the log file.
1204 */
1205 if( pWal->hdr.nPage ){
1206 sqlite3_log(SQLITE_OK, "Recovered %d frames from WAL file %s",
1207 pWal->hdr.nPage, pWal->zWalName
1208 );
1209 }
dan576bc322010-05-06 18:04:50 +00001210 }
drh73b64e42010-05-30 19:55:15 +00001211
1212recovery_error:
drhc74c3332010-05-31 12:15:19 +00001213 WALTRACE(("WAL%p: recovery %s\n", pWal, rc ? "failed" : "ok"));
dand0aa3422010-05-31 16:41:53 +00001214 walUnlockExclusive(pWal, iLock, nLock);
dan7c246102010-04-12 19:00:29 +00001215 return rc;
1216}
1217
drha8e654e2010-05-04 17:38:42 +00001218/*
dan1018e902010-05-05 15:33:05 +00001219** Close an open wal-index.
drha8e654e2010-05-04 17:38:42 +00001220*/
dan1018e902010-05-05 15:33:05 +00001221static void walIndexClose(Wal *pWal, int isDelete){
dan8c408002010-11-01 17:38:24 +00001222 if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){
1223 int i;
1224 for(i=0; i<pWal->nWiData; i++){
1225 sqlite3_free((void *)pWal->apWiData[i]);
1226 pWal->apWiData[i] = 0;
1227 }
1228 }else{
1229 sqlite3OsShmUnmap(pWal->pDbFd, isDelete);
1230 }
drha8e654e2010-05-04 17:38:42 +00001231}
1232
dan7c246102010-04-12 19:00:29 +00001233/*
dan3e875ef2010-07-05 19:03:35 +00001234** Open a connection to the WAL file zWalName. The database file must
1235** already be opened on connection pDbFd. The buffer that zWalName points
1236** to must remain valid for the lifetime of the returned Wal* handle.
dan3de777f2010-04-17 12:31:37 +00001237**
1238** A SHARED lock should be held on the database file when this function
1239** is called. The purpose of this SHARED lock is to prevent any other
drh181e0912010-06-01 01:08:08 +00001240** client from unlinking the WAL or wal-index file. If another process
dan3de777f2010-04-17 12:31:37 +00001241** were to do this just after this client opened one of these files, the
1242** system would be badly broken.
danef378022010-05-04 11:06:03 +00001243**
1244** If the log file is successfully opened, SQLITE_OK is returned and
1245** *ppWal is set to point to a new WAL handle. If an error occurs,
1246** an SQLite error code is returned and *ppWal is left unmodified.
dan7c246102010-04-12 19:00:29 +00001247*/
drhc438efd2010-04-26 00:19:45 +00001248int sqlite3WalOpen(
drh7ed91f22010-04-29 22:34:07 +00001249 sqlite3_vfs *pVfs, /* vfs module to open wal and wal-index */
drhd9e5c4f2010-05-12 18:01:39 +00001250 sqlite3_file *pDbFd, /* The open database file */
dan3e875ef2010-07-05 19:03:35 +00001251 const char *zWalName, /* Name of the WAL file */
dan8c408002010-11-01 17:38:24 +00001252 int bNoShm, /* True to run in heap-memory mode */
drh85a83752011-05-16 21:00:27 +00001253 i64 mxWalSize, /* Truncate WAL to this size on reset */
drh7ed91f22010-04-29 22:34:07 +00001254 Wal **ppWal /* OUT: Allocated Wal handle */
dan7c246102010-04-12 19:00:29 +00001255){
danef378022010-05-04 11:06:03 +00001256 int rc; /* Return Code */
drh7ed91f22010-04-29 22:34:07 +00001257 Wal *pRet; /* Object to allocate and return */
dan7c246102010-04-12 19:00:29 +00001258 int flags; /* Flags passed to OsOpen() */
dan7c246102010-04-12 19:00:29 +00001259
dan3e875ef2010-07-05 19:03:35 +00001260 assert( zWalName && zWalName[0] );
drhd9e5c4f2010-05-12 18:01:39 +00001261 assert( pDbFd );
dan7c246102010-04-12 19:00:29 +00001262
drh1b78eaf2010-05-25 13:40:03 +00001263 /* In the amalgamation, the os_unix.c and os_win.c source files come before
1264 ** this source file. Verify that the #defines of the locking byte offsets
1265 ** in os_unix.c and os_win.c agree with the WALINDEX_LOCK_OFFSET value.
1266 */
1267#ifdef WIN_SHM_BASE
1268 assert( WIN_SHM_BASE==WALINDEX_LOCK_OFFSET );
1269#endif
1270#ifdef UNIX_SHM_BASE
1271 assert( UNIX_SHM_BASE==WALINDEX_LOCK_OFFSET );
1272#endif
1273
1274
drh7ed91f22010-04-29 22:34:07 +00001275 /* Allocate an instance of struct Wal to return. */
1276 *ppWal = 0;
dan3e875ef2010-07-05 19:03:35 +00001277 pRet = (Wal*)sqlite3MallocZero(sizeof(Wal) + pVfs->szOsFile);
dan76ed3bc2010-05-03 17:18:24 +00001278 if( !pRet ){
1279 return SQLITE_NOMEM;
1280 }
1281
dan7c246102010-04-12 19:00:29 +00001282 pRet->pVfs = pVfs;
drhd9e5c4f2010-05-12 18:01:39 +00001283 pRet->pWalFd = (sqlite3_file *)&pRet[1];
1284 pRet->pDbFd = pDbFd;
drh73b64e42010-05-30 19:55:15 +00001285 pRet->readLock = -1;
drh85a83752011-05-16 21:00:27 +00001286 pRet->mxWalSize = mxWalSize;
dan3e875ef2010-07-05 19:03:35 +00001287 pRet->zWalName = zWalName;
dan8c408002010-11-01 17:38:24 +00001288 pRet->exclusiveMode = (bNoShm ? WAL_HEAPMEMORY_MODE: WAL_NORMAL_MODE);
dan7c246102010-04-12 19:00:29 +00001289
drh7ed91f22010-04-29 22:34:07 +00001290 /* Open file handle on the write-ahead log file. */
danddb0ac42010-07-14 14:48:58 +00001291 flags = (SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|SQLITE_OPEN_WAL);
danda9fe0c2010-07-13 18:44:03 +00001292 rc = sqlite3OsOpen(pVfs, zWalName, pRet->pWalFd, flags, &flags);
dan50833e32010-07-14 16:37:17 +00001293 if( rc==SQLITE_OK && flags&SQLITE_OPEN_READONLY ){
drh66dfec8b2011-06-01 20:01:49 +00001294 pRet->readOnly = WAL_RDONLY;
dan50833e32010-07-14 16:37:17 +00001295 }
dan7c246102010-04-12 19:00:29 +00001296
dan7c246102010-04-12 19:00:29 +00001297 if( rc!=SQLITE_OK ){
dan1018e902010-05-05 15:33:05 +00001298 walIndexClose(pRet, 0);
drhd9e5c4f2010-05-12 18:01:39 +00001299 sqlite3OsClose(pRet->pWalFd);
danef378022010-05-04 11:06:03 +00001300 sqlite3_free(pRet);
1301 }else{
drh4eb02a42011-12-16 21:26:26 +00001302 int iDC = sqlite3OsDeviceCharacteristics(pRet->pWalFd);
1303 if( iDC & SQLITE_IOCAP_SEQUENTIAL ){ pRet->noSyncHeader = 1; }
danef378022010-05-04 11:06:03 +00001304 *ppWal = pRet;
drhc74c3332010-05-31 12:15:19 +00001305 WALTRACE(("WAL%d: opened\n", pRet));
dan7c246102010-04-12 19:00:29 +00001306 }
dan7c246102010-04-12 19:00:29 +00001307 return rc;
1308}
1309
drha2a42012010-05-18 18:01:08 +00001310/*
drh85a83752011-05-16 21:00:27 +00001311** Change the size to which the WAL file is trucated on each reset.
1312*/
1313void sqlite3WalLimit(Wal *pWal, i64 iLimit){
1314 if( pWal ) pWal->mxWalSize = iLimit;
1315}
1316
1317/*
drha2a42012010-05-18 18:01:08 +00001318** Find the smallest page number out of all pages held in the WAL that
1319** has not been returned by any prior invocation of this method on the
1320** same WalIterator object. Write into *piFrame the frame index where
1321** that page was last written into the WAL. Write into *piPage the page
1322** number.
1323**
1324** Return 0 on success. If there are no pages in the WAL with a page
1325** number larger than *piPage, then return 1.
1326*/
drh7ed91f22010-04-29 22:34:07 +00001327static int walIteratorNext(
1328 WalIterator *p, /* Iterator */
drha2a42012010-05-18 18:01:08 +00001329 u32 *piPage, /* OUT: The page number of the next page */
1330 u32 *piFrame /* OUT: Wal frame index of next page */
dan7c246102010-04-12 19:00:29 +00001331){
drha2a42012010-05-18 18:01:08 +00001332 u32 iMin; /* Result pgno must be greater than iMin */
1333 u32 iRet = 0xFFFFFFFF; /* 0xffffffff is never a valid page number */
1334 int i; /* For looping through segments */
dan7c246102010-04-12 19:00:29 +00001335
drha2a42012010-05-18 18:01:08 +00001336 iMin = p->iPrior;
1337 assert( iMin<0xffffffff );
dan7c246102010-04-12 19:00:29 +00001338 for(i=p->nSegment-1; i>=0; i--){
drh7ed91f22010-04-29 22:34:07 +00001339 struct WalSegment *pSegment = &p->aSegment[i];
dan13a3cb82010-06-11 19:04:21 +00001340 while( pSegment->iNext<pSegment->nEntry ){
drha2a42012010-05-18 18:01:08 +00001341 u32 iPg = pSegment->aPgno[pSegment->aIndex[pSegment->iNext]];
dan7c246102010-04-12 19:00:29 +00001342 if( iPg>iMin ){
1343 if( iPg<iRet ){
1344 iRet = iPg;
dan13a3cb82010-06-11 19:04:21 +00001345 *piFrame = pSegment->iZero + pSegment->aIndex[pSegment->iNext];
dan7c246102010-04-12 19:00:29 +00001346 }
1347 break;
1348 }
1349 pSegment->iNext++;
1350 }
dan7c246102010-04-12 19:00:29 +00001351 }
1352
drha2a42012010-05-18 18:01:08 +00001353 *piPage = p->iPrior = iRet;
dan7c246102010-04-12 19:00:29 +00001354 return (iRet==0xFFFFFFFF);
1355}
1356
danf544b4c2010-06-25 11:35:52 +00001357/*
1358** This function merges two sorted lists into a single sorted list.
drhd9c9b782010-12-15 21:02:06 +00001359**
1360** aLeft[] and aRight[] are arrays of indices. The sort key is
1361** aContent[aLeft[]] and aContent[aRight[]]. Upon entry, the following
1362** is guaranteed for all J<K:
1363**
1364** aContent[aLeft[J]] < aContent[aLeft[K]]
1365** aContent[aRight[J]] < aContent[aRight[K]]
1366**
1367** This routine overwrites aRight[] with a new (probably longer) sequence
1368** of indices such that the aRight[] contains every index that appears in
1369** either aLeft[] or the old aRight[] and such that the second condition
1370** above is still met.
1371**
1372** The aContent[aLeft[X]] values will be unique for all X. And the
1373** aContent[aRight[X]] values will be unique too. But there might be
1374** one or more combinations of X and Y such that
1375**
1376** aLeft[X]!=aRight[Y] && aContent[aLeft[X]] == aContent[aRight[Y]]
1377**
1378** When that happens, omit the aLeft[X] and use the aRight[Y] index.
danf544b4c2010-06-25 11:35:52 +00001379*/
1380static void walMerge(
drhd9c9b782010-12-15 21:02:06 +00001381 const u32 *aContent, /* Pages in wal - keys for the sort */
danf544b4c2010-06-25 11:35:52 +00001382 ht_slot *aLeft, /* IN: Left hand input list */
1383 int nLeft, /* IN: Elements in array *paLeft */
1384 ht_slot **paRight, /* IN/OUT: Right hand input list */
1385 int *pnRight, /* IN/OUT: Elements in *paRight */
1386 ht_slot *aTmp /* Temporary buffer */
1387){
1388 int iLeft = 0; /* Current index in aLeft */
1389 int iRight = 0; /* Current index in aRight */
1390 int iOut = 0; /* Current index in output buffer */
1391 int nRight = *pnRight;
1392 ht_slot *aRight = *paRight;
dan7c246102010-04-12 19:00:29 +00001393
danf544b4c2010-06-25 11:35:52 +00001394 assert( nLeft>0 && nRight>0 );
1395 while( iRight<nRight || iLeft<nLeft ){
1396 ht_slot logpage;
1397 Pgno dbpage;
1398
1399 if( (iLeft<nLeft)
1400 && (iRight>=nRight || aContent[aLeft[iLeft]]<aContent[aRight[iRight]])
1401 ){
1402 logpage = aLeft[iLeft++];
1403 }else{
1404 logpage = aRight[iRight++];
1405 }
1406 dbpage = aContent[logpage];
1407
1408 aTmp[iOut++] = logpage;
1409 if( iLeft<nLeft && aContent[aLeft[iLeft]]==dbpage ) iLeft++;
1410
1411 assert( iLeft>=nLeft || aContent[aLeft[iLeft]]>dbpage );
1412 assert( iRight>=nRight || aContent[aRight[iRight]]>dbpage );
1413 }
1414
1415 *paRight = aLeft;
1416 *pnRight = iOut;
1417 memcpy(aLeft, aTmp, sizeof(aTmp[0])*iOut);
1418}
1419
1420/*
drhd9c9b782010-12-15 21:02:06 +00001421** Sort the elements in list aList using aContent[] as the sort key.
1422** Remove elements with duplicate keys, preferring to keep the
1423** larger aList[] values.
1424**
1425** The aList[] entries are indices into aContent[]. The values in
1426** aList[] are to be sorted so that for all J<K:
1427**
1428** aContent[aList[J]] < aContent[aList[K]]
1429**
1430** For any X and Y such that
1431**
1432** aContent[aList[X]] == aContent[aList[Y]]
1433**
1434** Keep the larger of the two values aList[X] and aList[Y] and discard
1435** the smaller.
danf544b4c2010-06-25 11:35:52 +00001436*/
dan13a3cb82010-06-11 19:04:21 +00001437static void walMergesort(
drhd9c9b782010-12-15 21:02:06 +00001438 const u32 *aContent, /* Pages in wal */
dan067f3162010-06-14 10:30:12 +00001439 ht_slot *aBuffer, /* Buffer of at least *pnList items to use */
1440 ht_slot *aList, /* IN/OUT: List to sort */
drha2a42012010-05-18 18:01:08 +00001441 int *pnList /* IN/OUT: Number of elements in aList[] */
1442){
danf544b4c2010-06-25 11:35:52 +00001443 struct Sublist {
1444 int nList; /* Number of elements in aList */
1445 ht_slot *aList; /* Pointer to sub-list content */
1446 };
drha2a42012010-05-18 18:01:08 +00001447
danf544b4c2010-06-25 11:35:52 +00001448 const int nList = *pnList; /* Size of input list */
drhff828942010-06-26 21:34:06 +00001449 int nMerge = 0; /* Number of elements in list aMerge */
1450 ht_slot *aMerge = 0; /* List to be merged */
danf544b4c2010-06-25 11:35:52 +00001451 int iList; /* Index into input list */
drh7d113eb2010-06-26 20:00:54 +00001452 int iSub = 0; /* Index into aSub array */
danf544b4c2010-06-25 11:35:52 +00001453 struct Sublist aSub[13]; /* Array of sub-lists */
drha2a42012010-05-18 18:01:08 +00001454
danf544b4c2010-06-25 11:35:52 +00001455 memset(aSub, 0, sizeof(aSub));
1456 assert( nList<=HASHTABLE_NPAGE && nList>0 );
1457 assert( HASHTABLE_NPAGE==(1<<(ArraySize(aSub)-1)) );
drha2a42012010-05-18 18:01:08 +00001458
danf544b4c2010-06-25 11:35:52 +00001459 for(iList=0; iList<nList; iList++){
1460 nMerge = 1;
1461 aMerge = &aList[iList];
1462 for(iSub=0; iList & (1<<iSub); iSub++){
1463 struct Sublist *p = &aSub[iSub];
1464 assert( p->aList && p->nList<=(1<<iSub) );
danbdf1e242010-06-25 15:16:25 +00001465 assert( p->aList==&aList[iList&~((2<<iSub)-1)] );
danf544b4c2010-06-25 11:35:52 +00001466 walMerge(aContent, p->aList, p->nList, &aMerge, &nMerge, aBuffer);
drha2a42012010-05-18 18:01:08 +00001467 }
danf544b4c2010-06-25 11:35:52 +00001468 aSub[iSub].aList = aMerge;
1469 aSub[iSub].nList = nMerge;
drha2a42012010-05-18 18:01:08 +00001470 }
1471
danf544b4c2010-06-25 11:35:52 +00001472 for(iSub++; iSub<ArraySize(aSub); iSub++){
1473 if( nList & (1<<iSub) ){
1474 struct Sublist *p = &aSub[iSub];
danbdf1e242010-06-25 15:16:25 +00001475 assert( p->nList<=(1<<iSub) );
1476 assert( p->aList==&aList[nList&~((2<<iSub)-1)] );
danf544b4c2010-06-25 11:35:52 +00001477 walMerge(aContent, p->aList, p->nList, &aMerge, &nMerge, aBuffer);
1478 }
1479 }
1480 assert( aMerge==aList );
1481 *pnList = nMerge;
1482
drha2a42012010-05-18 18:01:08 +00001483#ifdef SQLITE_DEBUG
1484 {
1485 int i;
1486 for(i=1; i<*pnList; i++){
1487 assert( aContent[aList[i]] > aContent[aList[i-1]] );
1488 }
1489 }
1490#endif
1491}
1492
dan5d656852010-06-14 07:53:26 +00001493/*
1494** Free an iterator allocated by walIteratorInit().
1495*/
1496static void walIteratorFree(WalIterator *p){
danbdf1e242010-06-25 15:16:25 +00001497 sqlite3ScratchFree(p);
dan5d656852010-06-14 07:53:26 +00001498}
1499
drha2a42012010-05-18 18:01:08 +00001500/*
danbdf1e242010-06-25 15:16:25 +00001501** Construct a WalInterator object that can be used to loop over all
1502** pages in the WAL in ascending order. The caller must hold the checkpoint
drhd9c9b782010-12-15 21:02:06 +00001503** lock.
drha2a42012010-05-18 18:01:08 +00001504**
1505** On success, make *pp point to the newly allocated WalInterator object
danbdf1e242010-06-25 15:16:25 +00001506** return SQLITE_OK. Otherwise, return an error code. If this routine
1507** returns an error, the value of *pp is undefined.
drha2a42012010-05-18 18:01:08 +00001508**
1509** The calling routine should invoke walIteratorFree() to destroy the
danbdf1e242010-06-25 15:16:25 +00001510** WalIterator object when it has finished with it.
drha2a42012010-05-18 18:01:08 +00001511*/
1512static int walIteratorInit(Wal *pWal, WalIterator **pp){
dan067f3162010-06-14 10:30:12 +00001513 WalIterator *p; /* Return value */
1514 int nSegment; /* Number of segments to merge */
1515 u32 iLast; /* Last frame in log */
1516 int nByte; /* Number of bytes to allocate */
1517 int i; /* Iterator variable */
1518 ht_slot *aTmp; /* Temp space used by merge-sort */
danbdf1e242010-06-25 15:16:25 +00001519 int rc = SQLITE_OK; /* Return Code */
drha2a42012010-05-18 18:01:08 +00001520
danbdf1e242010-06-25 15:16:25 +00001521 /* This routine only runs while holding the checkpoint lock. And
1522 ** it only runs if there is actually content in the log (mxFrame>0).
drha2a42012010-05-18 18:01:08 +00001523 */
danbdf1e242010-06-25 15:16:25 +00001524 assert( pWal->ckptLock && pWal->hdr.mxFrame>0 );
dan13a3cb82010-06-11 19:04:21 +00001525 iLast = pWal->hdr.mxFrame;
drha2a42012010-05-18 18:01:08 +00001526
danbdf1e242010-06-25 15:16:25 +00001527 /* Allocate space for the WalIterator object. */
dan13a3cb82010-06-11 19:04:21 +00001528 nSegment = walFramePage(iLast) + 1;
1529 nByte = sizeof(WalIterator)
dan52d6fc02010-06-25 16:34:32 +00001530 + (nSegment-1)*sizeof(struct WalSegment)
1531 + iLast*sizeof(ht_slot);
danbdf1e242010-06-25 15:16:25 +00001532 p = (WalIterator *)sqlite3ScratchMalloc(nByte);
dan8f6097c2010-05-06 07:43:58 +00001533 if( !p ){
drha2a42012010-05-18 18:01:08 +00001534 return SQLITE_NOMEM;
1535 }
1536 memset(p, 0, nByte);
drha2a42012010-05-18 18:01:08 +00001537 p->nSegment = nSegment;
danbdf1e242010-06-25 15:16:25 +00001538
1539 /* Allocate temporary space used by the merge-sort routine. This block
1540 ** of memory will be freed before this function returns.
1541 */
dan52d6fc02010-06-25 16:34:32 +00001542 aTmp = (ht_slot *)sqlite3ScratchMalloc(
1543 sizeof(ht_slot) * (iLast>HASHTABLE_NPAGE?HASHTABLE_NPAGE:iLast)
1544 );
danbdf1e242010-06-25 15:16:25 +00001545 if( !aTmp ){
1546 rc = SQLITE_NOMEM;
1547 }
1548
1549 for(i=0; rc==SQLITE_OK && i<nSegment; i++){
dan067f3162010-06-14 10:30:12 +00001550 volatile ht_slot *aHash;
dan13a3cb82010-06-11 19:04:21 +00001551 u32 iZero;
dan13a3cb82010-06-11 19:04:21 +00001552 volatile u32 *aPgno;
1553
dan4280eb32010-06-12 12:02:35 +00001554 rc = walHashGet(pWal, i, &aHash, &aPgno, &iZero);
danbdf1e242010-06-25 15:16:25 +00001555 if( rc==SQLITE_OK ){
dan52d6fc02010-06-25 16:34:32 +00001556 int j; /* Counter variable */
1557 int nEntry; /* Number of entries in this segment */
1558 ht_slot *aIndex; /* Sorted index for this segment */
1559
danbdf1e242010-06-25 15:16:25 +00001560 aPgno++;
drh519426a2010-07-09 03:19:07 +00001561 if( (i+1)==nSegment ){
1562 nEntry = (int)(iLast - iZero);
1563 }else{
shaneh55897962010-07-09 12:57:53 +00001564 nEntry = (int)((u32*)aHash - (u32*)aPgno);
drh519426a2010-07-09 03:19:07 +00001565 }
dan52d6fc02010-06-25 16:34:32 +00001566 aIndex = &((ht_slot *)&p->aSegment[p->nSegment])[iZero];
danbdf1e242010-06-25 15:16:25 +00001567 iZero++;
1568
danbdf1e242010-06-25 15:16:25 +00001569 for(j=0; j<nEntry; j++){
shaneh5eba1f62010-07-02 17:05:03 +00001570 aIndex[j] = (ht_slot)j;
danbdf1e242010-06-25 15:16:25 +00001571 }
1572 walMergesort((u32 *)aPgno, aTmp, aIndex, &nEntry);
1573 p->aSegment[i].iZero = iZero;
1574 p->aSegment[i].nEntry = nEntry;
1575 p->aSegment[i].aIndex = aIndex;
1576 p->aSegment[i].aPgno = (u32 *)aPgno;
dan13a3cb82010-06-11 19:04:21 +00001577 }
dan7c246102010-04-12 19:00:29 +00001578 }
danbdf1e242010-06-25 15:16:25 +00001579 sqlite3ScratchFree(aTmp);
dan7c246102010-04-12 19:00:29 +00001580
danbdf1e242010-06-25 15:16:25 +00001581 if( rc!=SQLITE_OK ){
1582 walIteratorFree(p);
1583 }
dan8f6097c2010-05-06 07:43:58 +00001584 *pp = p;
danbdf1e242010-06-25 15:16:25 +00001585 return rc;
dan7c246102010-04-12 19:00:29 +00001586}
1587
dan7c246102010-04-12 19:00:29 +00001588/*
dana58f26f2010-11-16 18:56:51 +00001589** Attempt to obtain the exclusive WAL lock defined by parameters lockIdx and
1590** n. If the attempt fails and parameter xBusy is not NULL, then it is a
1591** busy-handler function. Invoke it and retry the lock until either the
1592** lock is successfully obtained or the busy-handler returns 0.
1593*/
1594static int walBusyLock(
1595 Wal *pWal, /* WAL connection */
1596 int (*xBusy)(void*), /* Function to call when busy */
1597 void *pBusyArg, /* Context argument for xBusyHandler */
1598 int lockIdx, /* Offset of first byte to lock */
1599 int n /* Number of bytes to lock */
1600){
1601 int rc;
1602 do {
1603 rc = walLockExclusive(pWal, lockIdx, n);
1604 }while( xBusy && rc==SQLITE_BUSY && xBusy(pBusyArg) );
1605 return rc;
1606}
1607
1608/*
danf2b8dd52010-11-18 19:28:01 +00001609** The cache of the wal-index header must be valid to call this function.
1610** Return the page-size in bytes used by the database.
1611*/
1612static int walPagesize(Wal *pWal){
1613 return (pWal->hdr.szPage&0xfe00) + ((pWal->hdr.szPage&0x0001)<<16);
1614}
1615
1616/*
drh73b64e42010-05-30 19:55:15 +00001617** Copy as much content as we can from the WAL back into the database file
1618** in response to an sqlite3_wal_checkpoint() request or the equivalent.
1619**
1620** The amount of information copies from WAL to database might be limited
1621** by active readers. This routine will never overwrite a database page
1622** that a concurrent reader might be using.
1623**
1624** All I/O barrier operations (a.k.a fsyncs) occur in this routine when
1625** SQLite is in WAL-mode in synchronous=NORMAL. That means that if
1626** checkpoints are always run by a background thread or background
1627** process, foreground threads will never block on a lengthy fsync call.
1628**
1629** Fsync is called on the WAL before writing content out of the WAL and
1630** into the database. This ensures that if the new content is persistent
1631** in the WAL and can be recovered following a power-loss or hard reset.
1632**
1633** Fsync is also called on the database file if (and only if) the entire
1634** WAL content is copied into the database file. This second fsync makes
1635** it safe to delete the WAL since the new content will persist in the
1636** database file.
1637**
1638** This routine uses and updates the nBackfill field of the wal-index header.
1639** This is the only routine tha will increase the value of nBackfill.
1640** (A WAL reset or recovery will revert nBackfill to zero, but not increase
1641** its value.)
1642**
1643** The caller must be holding sufficient locks to ensure that no other
1644** checkpoint is running (in any other thread or process) at the same
1645** time.
dan7c246102010-04-12 19:00:29 +00001646*/
drh7ed91f22010-04-29 22:34:07 +00001647static int walCheckpoint(
1648 Wal *pWal, /* Wal connection */
dancdc1f042010-11-18 12:11:05 +00001649 int eMode, /* One of PASSIVE, FULL or RESTART */
danf2b8dd52010-11-18 19:28:01 +00001650 int (*xBusyCall)(void*), /* Function to call when busy */
dana58f26f2010-11-16 18:56:51 +00001651 void *pBusyArg, /* Context argument for xBusyHandler */
danc5118782010-04-17 17:34:41 +00001652 int sync_flags, /* Flags for OsSync() (or 0) */
dan9c5e3682011-02-07 15:12:12 +00001653 u8 *zBuf /* Temporary buffer to use */
dan7c246102010-04-12 19:00:29 +00001654){
1655 int rc; /* Return code */
drhb2eced52010-08-12 02:41:12 +00001656 int szPage; /* Database page-size */
drh7ed91f22010-04-29 22:34:07 +00001657 WalIterator *pIter = 0; /* Wal iterator context */
dan7c246102010-04-12 19:00:29 +00001658 u32 iDbpage = 0; /* Next database page to write */
drh7ed91f22010-04-29 22:34:07 +00001659 u32 iFrame = 0; /* Wal frame containing data for iDbpage */
drh73b64e42010-05-30 19:55:15 +00001660 u32 mxSafeFrame; /* Max frame that can be backfilled */
dan502019c2010-07-28 14:26:17 +00001661 u32 mxPage; /* Max database page to write */
drh73b64e42010-05-30 19:55:15 +00001662 int i; /* Loop counter */
drh73b64e42010-05-30 19:55:15 +00001663 volatile WalCkptInfo *pInfo; /* The checkpoint status information */
danf2b8dd52010-11-18 19:28:01 +00001664 int (*xBusy)(void*) = 0; /* Function to call when waiting for locks */
dan7c246102010-04-12 19:00:29 +00001665
danf2b8dd52010-11-18 19:28:01 +00001666 szPage = walPagesize(pWal);
drh9b78f792010-08-14 21:21:24 +00001667 testcase( szPage<=32768 );
1668 testcase( szPage>=65536 );
drh7d208442010-12-16 02:06:29 +00001669 pInfo = walCkptInfo(pWal);
1670 if( pInfo->nBackfill>=pWal->hdr.mxFrame ) return SQLITE_OK;
danf544b4c2010-06-25 11:35:52 +00001671
dan7c246102010-04-12 19:00:29 +00001672 /* Allocate the iterator */
dan8f6097c2010-05-06 07:43:58 +00001673 rc = walIteratorInit(pWal, &pIter);
danf544b4c2010-06-25 11:35:52 +00001674 if( rc!=SQLITE_OK ){
danbdf1e242010-06-25 15:16:25 +00001675 return rc;
danb6e099a2010-05-04 14:47:39 +00001676 }
danf544b4c2010-06-25 11:35:52 +00001677 assert( pIter );
danb6e099a2010-05-04 14:47:39 +00001678
danf2b8dd52010-11-18 19:28:01 +00001679 if( eMode!=SQLITE_CHECKPOINT_PASSIVE ) xBusy = xBusyCall;
danb6e099a2010-05-04 14:47:39 +00001680
drh73b64e42010-05-30 19:55:15 +00001681 /* Compute in mxSafeFrame the index of the last frame of the WAL that is
1682 ** safe to write into the database. Frames beyond mxSafeFrame might
1683 ** overwrite database pages that are in use by active readers and thus
1684 ** cannot be backfilled from the WAL.
1685 */
dand54ff602010-05-31 11:16:30 +00001686 mxSafeFrame = pWal->hdr.mxFrame;
dan502019c2010-07-28 14:26:17 +00001687 mxPage = pWal->hdr.nPage;
drh73b64e42010-05-30 19:55:15 +00001688 for(i=1; i<WAL_NREADER; i++){
1689 u32 y = pInfo->aReadMark[i];
danf2b8dd52010-11-18 19:28:01 +00001690 if( mxSafeFrame>y ){
dan83f42d12010-06-04 10:37:05 +00001691 assert( y<=pWal->hdr.mxFrame );
danf2b8dd52010-11-18 19:28:01 +00001692 rc = walBusyLock(pWal, xBusy, pBusyArg, WAL_READ_LOCK(i), 1);
dan83f42d12010-06-04 10:37:05 +00001693 if( rc==SQLITE_OK ){
drhdb7f6472010-06-09 14:45:12 +00001694 pInfo->aReadMark[i] = READMARK_NOT_USED;
drh73b64e42010-05-30 19:55:15 +00001695 walUnlockExclusive(pWal, WAL_READ_LOCK(i), 1);
drh2d37e1c2010-06-02 20:38:20 +00001696 }else if( rc==SQLITE_BUSY ){
drhdb7f6472010-06-09 14:45:12 +00001697 mxSafeFrame = y;
danf2b8dd52010-11-18 19:28:01 +00001698 xBusy = 0;
drh2d37e1c2010-06-02 20:38:20 +00001699 }else{
dan83f42d12010-06-04 10:37:05 +00001700 goto walcheckpoint_out;
drh73b64e42010-05-30 19:55:15 +00001701 }
1702 }
danc5118782010-04-17 17:34:41 +00001703 }
dan7c246102010-04-12 19:00:29 +00001704
drh73b64e42010-05-30 19:55:15 +00001705 if( pInfo->nBackfill<mxSafeFrame
dana58f26f2010-11-16 18:56:51 +00001706 && (rc = walBusyLock(pWal, xBusy, pBusyArg, WAL_READ_LOCK(0), 1))==SQLITE_OK
drh73b64e42010-05-30 19:55:15 +00001707 ){
dan502019c2010-07-28 14:26:17 +00001708 i64 nSize; /* Current size of database file */
drh73b64e42010-05-30 19:55:15 +00001709 u32 nBackfill = pInfo->nBackfill;
1710
1711 /* Sync the WAL to disk */
1712 if( sync_flags ){
1713 rc = sqlite3OsSync(pWal->pWalFd, sync_flags);
1714 }
1715
dan502019c2010-07-28 14:26:17 +00001716 /* If the database file may grow as a result of this checkpoint, hint
1717 ** about the eventual size of the db file to the VFS layer.
1718 */
dan007820d2010-08-09 07:51:40 +00001719 if( rc==SQLITE_OK ){
1720 i64 nReq = ((i64)mxPage * szPage);
1721 rc = sqlite3OsFileSize(pWal->pDbFd, &nSize);
1722 if( rc==SQLITE_OK && nSize<nReq ){
1723 sqlite3OsFileControl(pWal->pDbFd, SQLITE_FCNTL_SIZE_HINT, &nReq);
1724 }
dan502019c2010-07-28 14:26:17 +00001725 }
1726
drh73b64e42010-05-30 19:55:15 +00001727 /* Iterate through the contents of the WAL, copying data to the db file. */
1728 while( rc==SQLITE_OK && 0==walIteratorNext(pIter, &iDbpage, &iFrame) ){
drh3e8e7ec2010-07-07 13:43:19 +00001729 i64 iOffset;
dan13a3cb82010-06-11 19:04:21 +00001730 assert( walFramePgno(pWal, iFrame)==iDbpage );
dan502019c2010-07-28 14:26:17 +00001731 if( iFrame<=nBackfill || iFrame>mxSafeFrame || iDbpage>mxPage ) continue;
drh3e8e7ec2010-07-07 13:43:19 +00001732 iOffset = walFrameOffset(iFrame, szPage) + WAL_FRAME_HDRSIZE;
drh09b5dbc2010-07-07 14:35:58 +00001733 /* testcase( IS_BIG_INT(iOffset) ); // requires a 4GiB WAL file */
drh3e8e7ec2010-07-07 13:43:19 +00001734 rc = sqlite3OsRead(pWal->pWalFd, zBuf, szPage, iOffset);
1735 if( rc!=SQLITE_OK ) break;
1736 iOffset = (iDbpage-1)*(i64)szPage;
1737 testcase( IS_BIG_INT(iOffset) );
1738 rc = sqlite3OsWrite(pWal->pDbFd, zBuf, szPage, iOffset);
1739 if( rc!=SQLITE_OK ) break;
drh73b64e42010-05-30 19:55:15 +00001740 }
1741
1742 /* If work was actually accomplished... */
dand764c7d2010-06-04 11:56:22 +00001743 if( rc==SQLITE_OK ){
dan4280eb32010-06-12 12:02:35 +00001744 if( mxSafeFrame==walIndexHdr(pWal)->mxFrame ){
drh3e8e7ec2010-07-07 13:43:19 +00001745 i64 szDb = pWal->hdr.nPage*(i64)szPage;
1746 testcase( IS_BIG_INT(szDb) );
1747 rc = sqlite3OsTruncate(pWal->pDbFd, szDb);
drh73b64e42010-05-30 19:55:15 +00001748 if( rc==SQLITE_OK && sync_flags ){
1749 rc = sqlite3OsSync(pWal->pDbFd, sync_flags);
1750 }
1751 }
dand764c7d2010-06-04 11:56:22 +00001752 if( rc==SQLITE_OK ){
1753 pInfo->nBackfill = mxSafeFrame;
1754 }
drh73b64e42010-05-30 19:55:15 +00001755 }
1756
1757 /* Release the reader lock held while backfilling */
1758 walUnlockExclusive(pWal, WAL_READ_LOCK(0), 1);
dana58f26f2010-11-16 18:56:51 +00001759 }
1760
1761 if( rc==SQLITE_BUSY ){
drh34116ea2010-05-31 12:30:52 +00001762 /* Reset the return code so as not to report a checkpoint failure
dana58f26f2010-11-16 18:56:51 +00001763 ** just because there are active readers. */
drh34116ea2010-05-31 12:30:52 +00001764 rc = SQLITE_OK;
dan7c246102010-04-12 19:00:29 +00001765 }
1766
danf2b8dd52010-11-18 19:28:01 +00001767 /* If this is an SQLITE_CHECKPOINT_RESTART operation, and the entire wal
1768 ** file has been copied into the database file, then block until all
1769 ** readers have finished using the wal file. This ensures that the next
1770 ** process to write to the database restarts the wal file.
1771 */
1772 if( rc==SQLITE_OK && eMode!=SQLITE_CHECKPOINT_PASSIVE ){
dancdc1f042010-11-18 12:11:05 +00001773 assert( pWal->writeLock );
danf2b8dd52010-11-18 19:28:01 +00001774 if( pInfo->nBackfill<pWal->hdr.mxFrame ){
1775 rc = SQLITE_BUSY;
1776 }else if( eMode==SQLITE_CHECKPOINT_RESTART ){
1777 assert( mxSafeFrame==pWal->hdr.mxFrame );
1778 rc = walBusyLock(pWal, xBusy, pBusyArg, WAL_READ_LOCK(1), WAL_NREADER-1);
1779 if( rc==SQLITE_OK ){
1780 walUnlockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);
1781 }
dancdc1f042010-11-18 12:11:05 +00001782 }
1783 }
1784
dan83f42d12010-06-04 10:37:05 +00001785 walcheckpoint_out:
drh7ed91f22010-04-29 22:34:07 +00001786 walIteratorFree(pIter);
dan7c246102010-04-12 19:00:29 +00001787 return rc;
1788}
1789
1790/*
danf60b7f32011-12-16 13:24:27 +00001791** If the WAL file is currently larger than nMax bytes in size, truncate
1792** it to exactly nMax bytes. If an error occurs while doing so, ignore it.
drh8dd4afa2011-12-08 19:50:32 +00001793*/
danf60b7f32011-12-16 13:24:27 +00001794static void walLimitSize(Wal *pWal, i64 nMax){
1795 i64 sz;
1796 int rx;
1797 sqlite3BeginBenignMalloc();
1798 rx = sqlite3OsFileSize(pWal->pWalFd, &sz);
1799 if( rx==SQLITE_OK && (sz > nMax ) ){
1800 rx = sqlite3OsTruncate(pWal->pWalFd, nMax);
1801 }
1802 sqlite3EndBenignMalloc();
1803 if( rx ){
1804 sqlite3_log(rx, "cannot limit WAL size: %s", pWal->zWalName);
drh8dd4afa2011-12-08 19:50:32 +00001805 }
1806}
1807
1808/*
dan7c246102010-04-12 19:00:29 +00001809** Close a connection to a log file.
1810*/
drhc438efd2010-04-26 00:19:45 +00001811int sqlite3WalClose(
drh7ed91f22010-04-29 22:34:07 +00001812 Wal *pWal, /* Wal to close */
danc5118782010-04-17 17:34:41 +00001813 int sync_flags, /* Flags to pass to OsSync() (or 0) */
danb6e099a2010-05-04 14:47:39 +00001814 int nBuf,
1815 u8 *zBuf /* Buffer of at least nBuf bytes */
dan7c246102010-04-12 19:00:29 +00001816){
1817 int rc = SQLITE_OK;
drh7ed91f22010-04-29 22:34:07 +00001818 if( pWal ){
dan30c86292010-04-30 16:24:46 +00001819 int isDelete = 0; /* True to unlink wal and wal-index files */
1820
1821 /* If an EXCLUSIVE lock can be obtained on the database file (using the
1822 ** ordinary, rollback-mode locking methods, this guarantees that the
1823 ** connection associated with this log file is the only connection to
1824 ** the database. In this case checkpoint the database and unlink both
1825 ** the wal and wal-index files.
1826 **
1827 ** The EXCLUSIVE lock is not released before returning.
1828 */
drhd9e5c4f2010-05-12 18:01:39 +00001829 rc = sqlite3OsLock(pWal->pDbFd, SQLITE_LOCK_EXCLUSIVE);
dan30c86292010-04-30 16:24:46 +00001830 if( rc==SQLITE_OK ){
dan8c408002010-11-01 17:38:24 +00001831 if( pWal->exclusiveMode==WAL_NORMAL_MODE ){
1832 pWal->exclusiveMode = WAL_EXCLUSIVE_MODE;
1833 }
dancdc1f042010-11-18 12:11:05 +00001834 rc = sqlite3WalCheckpoint(
1835 pWal, SQLITE_CHECKPOINT_PASSIVE, 0, 0, sync_flags, nBuf, zBuf, 0, 0
1836 );
drheed42502011-12-16 15:38:52 +00001837 if( rc==SQLITE_OK ){
1838 int bPersist = -1;
1839 sqlite3OsFileControl(pWal->pDbFd, SQLITE_FCNTL_PERSIST_WAL, &bPersist);
1840 if( bPersist!=1 ){
1841 /* Try to delete the WAL file if the checkpoint completed and
1842 ** fsyned (rc==SQLITE_OK) and if we are not in persistent-wal
1843 ** mode (!bPersist) */
1844 isDelete = 1;
1845 }else if( pWal->mxWalSize>=0 ){
1846 /* Try to truncate the WAL file to zero bytes if the checkpoint
1847 ** completed and fsynced (rc==SQLITE_OK) and we are in persistent
1848 ** WAL mode (bPersist) and if the PRAGMA journal_size_limit is a
1849 ** non-negative value (pWal->mxWalSize>=0). Note that we truncate
1850 ** to zero bytes as truncating to the journal_size_limit might
1851 ** leave a corrupt WAL file on disk. */
1852 walLimitSize(pWal, 0);
1853 }
dan30c86292010-04-30 16:24:46 +00001854 }
dan30c86292010-04-30 16:24:46 +00001855 }
1856
dan1018e902010-05-05 15:33:05 +00001857 walIndexClose(pWal, isDelete);
drhd9e5c4f2010-05-12 18:01:39 +00001858 sqlite3OsClose(pWal->pWalFd);
dan30c86292010-04-30 16:24:46 +00001859 if( isDelete ){
drhd9e5c4f2010-05-12 18:01:39 +00001860 sqlite3OsDelete(pWal->pVfs, pWal->zWalName, 0);
dan30c86292010-04-30 16:24:46 +00001861 }
drhc74c3332010-05-31 12:15:19 +00001862 WALTRACE(("WAL%p: closed\n", pWal));
shaneh8a300f82010-07-02 18:15:31 +00001863 sqlite3_free((void *)pWal->apWiData);
drh7ed91f22010-04-29 22:34:07 +00001864 sqlite3_free(pWal);
dan7c246102010-04-12 19:00:29 +00001865 }
1866 return rc;
1867}
1868
1869/*
drha2a42012010-05-18 18:01:08 +00001870** Try to read the wal-index header. Return 0 on success and 1 if
1871** there is a problem.
1872**
1873** The wal-index is in shared memory. Another thread or process might
1874** be writing the header at the same time this procedure is trying to
1875** read it, which might result in inconsistency. A dirty read is detected
drh73b64e42010-05-30 19:55:15 +00001876** by verifying that both copies of the header are the same and also by
1877** a checksum on the header.
drha2a42012010-05-18 18:01:08 +00001878**
1879** If and only if the read is consistent and the header is different from
1880** pWal->hdr, then pWal->hdr is updated to the content of the new header
1881** and *pChanged is set to 1.
danb9bf16b2010-04-14 11:23:30 +00001882**
dan84670502010-05-07 05:46:23 +00001883** If the checksum cannot be verified return non-zero. If the header
1884** is read successfully and the checksum verified, return zero.
danb9bf16b2010-04-14 11:23:30 +00001885*/
drh7750ab42010-06-26 22:16:02 +00001886static int walIndexTryHdr(Wal *pWal, int *pChanged){
dan4280eb32010-06-12 12:02:35 +00001887 u32 aCksum[2]; /* Checksum on the header content */
1888 WalIndexHdr h1, h2; /* Two copies of the header content */
1889 WalIndexHdr volatile *aHdr; /* Header in shared memory */
danb9bf16b2010-04-14 11:23:30 +00001890
dan4280eb32010-06-12 12:02:35 +00001891 /* The first page of the wal-index must be mapped at this point. */
1892 assert( pWal->nWiData>0 && pWal->apWiData[0] );
drh79e6c782010-04-30 02:13:26 +00001893
drh6cef0cf2010-08-16 16:31:43 +00001894 /* Read the header. This might happen concurrently with a write to the
drh73b64e42010-05-30 19:55:15 +00001895 ** same area of shared memory on a different CPU in a SMP,
1896 ** meaning it is possible that an inconsistent snapshot is read
dan84670502010-05-07 05:46:23 +00001897 ** from the file. If this happens, return non-zero.
drhf0b20f82010-05-21 13:16:18 +00001898 **
1899 ** There are two copies of the header at the beginning of the wal-index.
1900 ** When reading, read [0] first then [1]. Writes are in the reverse order.
1901 ** Memory barriers are used to prevent the compiler or the hardware from
1902 ** reordering the reads and writes.
danb9bf16b2010-04-14 11:23:30 +00001903 */
dan4280eb32010-06-12 12:02:35 +00001904 aHdr = walIndexHdr(pWal);
1905 memcpy(&h1, (void *)&aHdr[0], sizeof(h1));
dan8c408002010-11-01 17:38:24 +00001906 walShmBarrier(pWal);
dan4280eb32010-06-12 12:02:35 +00001907 memcpy(&h2, (void *)&aHdr[1], sizeof(h2));
drh286a2882010-05-20 23:51:06 +00001908
drhf0b20f82010-05-21 13:16:18 +00001909 if( memcmp(&h1, &h2, sizeof(h1))!=0 ){
1910 return 1; /* Dirty read */
drh286a2882010-05-20 23:51:06 +00001911 }
drh4b82c382010-05-31 18:24:19 +00001912 if( h1.isInit==0 ){
drhf0b20f82010-05-21 13:16:18 +00001913 return 1; /* Malformed header - probably all zeros */
1914 }
danb8fd6c22010-05-24 10:39:36 +00001915 walChecksumBytes(1, (u8*)&h1, sizeof(h1)-sizeof(h1.aCksum), 0, aCksum);
drhf0b20f82010-05-21 13:16:18 +00001916 if( aCksum[0]!=h1.aCksum[0] || aCksum[1]!=h1.aCksum[1] ){
1917 return 1; /* Checksum does not match */
danb9bf16b2010-04-14 11:23:30 +00001918 }
1919
drhf0b20f82010-05-21 13:16:18 +00001920 if( memcmp(&pWal->hdr, &h1, sizeof(WalIndexHdr)) ){
dana8614692010-05-06 14:42:34 +00001921 *pChanged = 1;
drhf0b20f82010-05-21 13:16:18 +00001922 memcpy(&pWal->hdr, &h1, sizeof(WalIndexHdr));
drh9b78f792010-08-14 21:21:24 +00001923 pWal->szPage = (pWal->hdr.szPage&0xfe00) + ((pWal->hdr.szPage&0x0001)<<16);
1924 testcase( pWal->szPage<=32768 );
1925 testcase( pWal->szPage>=65536 );
danb9bf16b2010-04-14 11:23:30 +00001926 }
dan84670502010-05-07 05:46:23 +00001927
1928 /* The header was successfully read. Return zero. */
1929 return 0;
danb9bf16b2010-04-14 11:23:30 +00001930}
1931
1932/*
drha2a42012010-05-18 18:01:08 +00001933** Read the wal-index header from the wal-index and into pWal->hdr.
drha927e942010-06-24 02:46:48 +00001934** If the wal-header appears to be corrupt, try to reconstruct the
1935** wal-index from the WAL before returning.
drha2a42012010-05-18 18:01:08 +00001936**
1937** Set *pChanged to 1 if the wal-index header value in pWal->hdr is
1938** changed by this opertion. If pWal->hdr is unchanged, set *pChanged
1939** to 0.
1940**
drh7ed91f22010-04-29 22:34:07 +00001941** If the wal-index header is successfully read, return SQLITE_OK.
danb9bf16b2010-04-14 11:23:30 +00001942** Otherwise an SQLite error code.
1943*/
drh7ed91f22010-04-29 22:34:07 +00001944static int walIndexReadHdr(Wal *pWal, int *pChanged){
dan84670502010-05-07 05:46:23 +00001945 int rc; /* Return code */
drh73b64e42010-05-30 19:55:15 +00001946 int badHdr; /* True if a header read failed */
drha927e942010-06-24 02:46:48 +00001947 volatile u32 *page0; /* Chunk of wal-index containing header */
danb9bf16b2010-04-14 11:23:30 +00001948
dan4280eb32010-06-12 12:02:35 +00001949 /* Ensure that page 0 of the wal-index (the page that contains the
1950 ** wal-index header) is mapped. Return early if an error occurs here.
1951 */
dana8614692010-05-06 14:42:34 +00001952 assert( pChanged );
dan4280eb32010-06-12 12:02:35 +00001953 rc = walIndexPage(pWal, 0, &page0);
danc7991bd2010-05-05 19:04:59 +00001954 if( rc!=SQLITE_OK ){
1955 return rc;
dan4280eb32010-06-12 12:02:35 +00001956 };
1957 assert( page0 || pWal->writeLock==0 );
drh7ed91f22010-04-29 22:34:07 +00001958
dan4280eb32010-06-12 12:02:35 +00001959 /* If the first page of the wal-index has been mapped, try to read the
1960 ** wal-index header immediately, without holding any lock. This usually
1961 ** works, but may fail if the wal-index header is corrupt or currently
drha927e942010-06-24 02:46:48 +00001962 ** being modified by another thread or process.
danb9bf16b2010-04-14 11:23:30 +00001963 */
dan4280eb32010-06-12 12:02:35 +00001964 badHdr = (page0 ? walIndexTryHdr(pWal, pChanged) : 1);
drhbab7b912010-05-26 17:31:58 +00001965
drh73b64e42010-05-30 19:55:15 +00001966 /* If the first attempt failed, it might have been due to a race
drh66dfec8b2011-06-01 20:01:49 +00001967 ** with a writer. So get a WRITE lock and try again.
drh73b64e42010-05-30 19:55:15 +00001968 */
dand54ff602010-05-31 11:16:30 +00001969 assert( badHdr==0 || pWal->writeLock==0 );
dan4edc6bf2011-05-10 17:31:29 +00001970 if( badHdr ){
drh66dfec8b2011-06-01 20:01:49 +00001971 if( pWal->readOnly & WAL_SHM_RDONLY ){
dan4edc6bf2011-05-10 17:31:29 +00001972 if( SQLITE_OK==(rc = walLockShared(pWal, WAL_WRITE_LOCK)) ){
1973 walUnlockShared(pWal, WAL_WRITE_LOCK);
1974 rc = SQLITE_READONLY_RECOVERY;
drhbab7b912010-05-26 17:31:58 +00001975 }
dan4edc6bf2011-05-10 17:31:29 +00001976 }else if( SQLITE_OK==(rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1)) ){
1977 pWal->writeLock = 1;
1978 if( SQLITE_OK==(rc = walIndexPage(pWal, 0, &page0)) ){
1979 badHdr = walIndexTryHdr(pWal, pChanged);
1980 if( badHdr ){
1981 /* If the wal-index header is still malformed even while holding
1982 ** a WRITE lock, it can only mean that the header is corrupted and
1983 ** needs to be reconstructed. So run recovery to do exactly that.
1984 */
1985 rc = walIndexRecover(pWal);
1986 *pChanged = 1;
1987 }
1988 }
1989 pWal->writeLock = 0;
1990 walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1);
drhbab7b912010-05-26 17:31:58 +00001991 }
danb9bf16b2010-04-14 11:23:30 +00001992 }
1993
drha927e942010-06-24 02:46:48 +00001994 /* If the header is read successfully, check the version number to make
1995 ** sure the wal-index was not constructed with some future format that
1996 ** this version of SQLite cannot understand.
1997 */
1998 if( badHdr==0 && pWal->hdr.iVersion!=WALINDEX_MAX_VERSION ){
1999 rc = SQLITE_CANTOPEN_BKPT;
2000 }
2001
danb9bf16b2010-04-14 11:23:30 +00002002 return rc;
2003}
2004
2005/*
drh73b64e42010-05-30 19:55:15 +00002006** This is the value that walTryBeginRead returns when it needs to
2007** be retried.
dan7c246102010-04-12 19:00:29 +00002008*/
drh73b64e42010-05-30 19:55:15 +00002009#define WAL_RETRY (-1)
dan64d039e2010-04-13 19:27:31 +00002010
drh73b64e42010-05-30 19:55:15 +00002011/*
2012** Attempt to start a read transaction. This might fail due to a race or
2013** other transient condition. When that happens, it returns WAL_RETRY to
2014** indicate to the caller that it is safe to retry immediately.
2015**
drha927e942010-06-24 02:46:48 +00002016** On success return SQLITE_OK. On a permanent failure (such an
drh73b64e42010-05-30 19:55:15 +00002017** I/O error or an SQLITE_BUSY because another process is running
2018** recovery) return a positive error code.
2019**
drha927e942010-06-24 02:46:48 +00002020** The useWal parameter is true to force the use of the WAL and disable
2021** the case where the WAL is bypassed because it has been completely
2022** checkpointed. If useWal==0 then this routine calls walIndexReadHdr()
2023** to make a copy of the wal-index header into pWal->hdr. If the
2024** wal-index header has changed, *pChanged is set to 1 (as an indication
2025** to the caller that the local paget cache is obsolete and needs to be
2026** flushed.) When useWal==1, the wal-index header is assumed to already
2027** be loaded and the pChanged parameter is unused.
2028**
2029** The caller must set the cnt parameter to the number of prior calls to
2030** this routine during the current read attempt that returned WAL_RETRY.
2031** This routine will start taking more aggressive measures to clear the
2032** race conditions after multiple WAL_RETRY returns, and after an excessive
2033** number of errors will ultimately return SQLITE_PROTOCOL. The
2034** SQLITE_PROTOCOL return indicates that some other process has gone rogue
2035** and is not honoring the locking protocol. There is a vanishingly small
2036** chance that SQLITE_PROTOCOL could be returned because of a run of really
2037** bad luck when there is lots of contention for the wal-index, but that
2038** possibility is so small that it can be safely neglected, we believe.
2039**
drh73b64e42010-05-30 19:55:15 +00002040** On success, this routine obtains a read lock on
2041** WAL_READ_LOCK(pWal->readLock). The pWal->readLock integer is
2042** in the range 0 <= pWal->readLock < WAL_NREADER. If pWal->readLock==(-1)
2043** that means the Wal does not hold any read lock. The reader must not
2044** access any database page that is modified by a WAL frame up to and
2045** including frame number aReadMark[pWal->readLock]. The reader will
2046** use WAL frames up to and including pWal->hdr.mxFrame if pWal->readLock>0
2047** Or if pWal->readLock==0, then the reader will ignore the WAL
2048** completely and get all content directly from the database file.
drha927e942010-06-24 02:46:48 +00002049** If the useWal parameter is 1 then the WAL will never be ignored and
2050** this routine will always set pWal->readLock>0 on success.
drh73b64e42010-05-30 19:55:15 +00002051** When the read transaction is completed, the caller must release the
2052** lock on WAL_READ_LOCK(pWal->readLock) and set pWal->readLock to -1.
2053**
2054** This routine uses the nBackfill and aReadMark[] fields of the header
2055** to select a particular WAL_READ_LOCK() that strives to let the
2056** checkpoint process do as much work as possible. This routine might
2057** update values of the aReadMark[] array in the header, but if it does
2058** so it takes care to hold an exclusive lock on the corresponding
2059** WAL_READ_LOCK() while changing values.
2060*/
drhaab4c022010-06-02 14:45:51 +00002061static int walTryBeginRead(Wal *pWal, int *pChanged, int useWal, int cnt){
drh73b64e42010-05-30 19:55:15 +00002062 volatile WalCkptInfo *pInfo; /* Checkpoint information in wal-index */
2063 u32 mxReadMark; /* Largest aReadMark[] value */
2064 int mxI; /* Index of largest aReadMark[] value */
2065 int i; /* Loop counter */
dan13a3cb82010-06-11 19:04:21 +00002066 int rc = SQLITE_OK; /* Return code */
dan64d039e2010-04-13 19:27:31 +00002067
drh61e4ace2010-05-31 20:28:37 +00002068 assert( pWal->readLock<0 ); /* Not currently locked */
drh73b64e42010-05-30 19:55:15 +00002069
drh658d76c2011-02-19 15:22:14 +00002070 /* Take steps to avoid spinning forever if there is a protocol error.
2071 **
2072 ** Circumstances that cause a RETRY should only last for the briefest
2073 ** instances of time. No I/O or other system calls are done while the
2074 ** locks are held, so the locks should not be held for very long. But
2075 ** if we are unlucky, another process that is holding a lock might get
2076 ** paged out or take a page-fault that is time-consuming to resolve,
2077 ** during the few nanoseconds that it is holding the lock. In that case,
2078 ** it might take longer than normal for the lock to free.
2079 **
2080 ** After 5 RETRYs, we begin calling sqlite3OsSleep(). The first few
2081 ** calls to sqlite3OsSleep() have a delay of 1 microsecond. Really this
2082 ** is more of a scheduler yield than an actual delay. But on the 10th
2083 ** an subsequent retries, the delays start becoming longer and longer,
2084 ** so that on the 100th (and last) RETRY we delay for 21 milliseconds.
2085 ** The total delay time before giving up is less than 1 second.
2086 */
drhaab4c022010-06-02 14:45:51 +00002087 if( cnt>5 ){
drh658d76c2011-02-19 15:22:14 +00002088 int nDelay = 1; /* Pause time in microseconds */
drh03c69672011-02-19 23:18:12 +00002089 if( cnt>100 ){
2090 VVA_ONLY( pWal->lockError = 1; )
2091 return SQLITE_PROTOCOL;
2092 }
drh658d76c2011-02-19 15:22:14 +00002093 if( cnt>=10 ) nDelay = (cnt-9)*238; /* Max delay 21ms. Total delay 996ms */
2094 sqlite3OsSleep(pWal->pVfs, nDelay);
drhaab4c022010-06-02 14:45:51 +00002095 }
2096
drh73b64e42010-05-30 19:55:15 +00002097 if( !useWal ){
drh7ed91f22010-04-29 22:34:07 +00002098 rc = walIndexReadHdr(pWal, pChanged);
drh73b64e42010-05-30 19:55:15 +00002099 if( rc==SQLITE_BUSY ){
2100 /* If there is not a recovery running in another thread or process
2101 ** then convert BUSY errors to WAL_RETRY. If recovery is known to
2102 ** be running, convert BUSY to BUSY_RECOVERY. There is a race here
2103 ** which might cause WAL_RETRY to be returned even if BUSY_RECOVERY
2104 ** would be technically correct. But the race is benign since with
2105 ** WAL_RETRY this routine will be called again and will probably be
2106 ** right on the second iteration.
2107 */
dan7d4514a2010-07-15 17:54:14 +00002108 if( pWal->apWiData[0]==0 ){
2109 /* This branch is taken when the xShmMap() method returns SQLITE_BUSY.
2110 ** We assume this is a transient condition, so return WAL_RETRY. The
2111 ** xShmMap() implementation used by the default unix and win32 VFS
2112 ** modules may return SQLITE_BUSY due to a race condition in the
2113 ** code that determines whether or not the shared-memory region
2114 ** must be zeroed before the requested page is returned.
2115 */
2116 rc = WAL_RETRY;
2117 }else if( SQLITE_OK==(rc = walLockShared(pWal, WAL_RECOVER_LOCK)) ){
drh73b64e42010-05-30 19:55:15 +00002118 walUnlockShared(pWal, WAL_RECOVER_LOCK);
2119 rc = WAL_RETRY;
2120 }else if( rc==SQLITE_BUSY ){
2121 rc = SQLITE_BUSY_RECOVERY;
2122 }
2123 }
drha927e942010-06-24 02:46:48 +00002124 if( rc!=SQLITE_OK ){
2125 return rc;
2126 }
drh73b64e42010-05-30 19:55:15 +00002127 }
2128
dan13a3cb82010-06-11 19:04:21 +00002129 pInfo = walCkptInfo(pWal);
drh73b64e42010-05-30 19:55:15 +00002130 if( !useWal && pInfo->nBackfill==pWal->hdr.mxFrame ){
2131 /* The WAL has been completely backfilled (or it is empty).
2132 ** and can be safely ignored.
2133 */
2134 rc = walLockShared(pWal, WAL_READ_LOCK(0));
dan8c408002010-11-01 17:38:24 +00002135 walShmBarrier(pWal);
drh73b64e42010-05-30 19:55:15 +00002136 if( rc==SQLITE_OK ){
dan4280eb32010-06-12 12:02:35 +00002137 if( memcmp((void *)walIndexHdr(pWal), &pWal->hdr, sizeof(WalIndexHdr)) ){
dan493cc592010-06-05 18:12:23 +00002138 /* It is not safe to allow the reader to continue here if frames
2139 ** may have been appended to the log before READ_LOCK(0) was obtained.
2140 ** When holding READ_LOCK(0), the reader ignores the entire log file,
2141 ** which implies that the database file contains a trustworthy
2142 ** snapshoT. Since holding READ_LOCK(0) prevents a checkpoint from
2143 ** happening, this is usually correct.
2144 **
2145 ** However, if frames have been appended to the log (or if the log
2146 ** is wrapped and written for that matter) before the READ_LOCK(0)
2147 ** is obtained, that is not necessarily true. A checkpointer may
2148 ** have started to backfill the appended frames but crashed before
2149 ** it finished. Leaving a corrupt image in the database file.
2150 */
drh73b64e42010-05-30 19:55:15 +00002151 walUnlockShared(pWal, WAL_READ_LOCK(0));
2152 return WAL_RETRY;
2153 }
2154 pWal->readLock = 0;
2155 return SQLITE_OK;
2156 }else if( rc!=SQLITE_BUSY ){
2157 return rc;
dan64d039e2010-04-13 19:27:31 +00002158 }
dan7c246102010-04-12 19:00:29 +00002159 }
danba515902010-04-30 09:32:06 +00002160
drh73b64e42010-05-30 19:55:15 +00002161 /* If we get this far, it means that the reader will want to use
2162 ** the WAL to get at content from recent commits. The job now is
2163 ** to select one of the aReadMark[] entries that is closest to
2164 ** but not exceeding pWal->hdr.mxFrame and lock that entry.
2165 */
2166 mxReadMark = 0;
2167 mxI = 0;
2168 for(i=1; i<WAL_NREADER; i++){
2169 u32 thisMark = pInfo->aReadMark[i];
drhdb7f6472010-06-09 14:45:12 +00002170 if( mxReadMark<=thisMark && thisMark<=pWal->hdr.mxFrame ){
2171 assert( thisMark!=READMARK_NOT_USED );
drh73b64e42010-05-30 19:55:15 +00002172 mxReadMark = thisMark;
2173 mxI = i;
2174 }
2175 }
drh658d76c2011-02-19 15:22:14 +00002176 /* There was once an "if" here. The extra "{" is to preserve indentation. */
2177 {
drh66dfec8b2011-06-01 20:01:49 +00002178 if( (pWal->readOnly & WAL_SHM_RDONLY)==0
2179 && (mxReadMark<pWal->hdr.mxFrame || mxI==0)
2180 ){
dand54ff602010-05-31 11:16:30 +00002181 for(i=1; i<WAL_NREADER; i++){
drh73b64e42010-05-30 19:55:15 +00002182 rc = walLockExclusive(pWal, WAL_READ_LOCK(i), 1);
2183 if( rc==SQLITE_OK ){
drhdb7f6472010-06-09 14:45:12 +00002184 mxReadMark = pInfo->aReadMark[i] = pWal->hdr.mxFrame;
drh73b64e42010-05-30 19:55:15 +00002185 mxI = i;
2186 walUnlockExclusive(pWal, WAL_READ_LOCK(i), 1);
2187 break;
drh38933f22010-06-02 15:43:18 +00002188 }else if( rc!=SQLITE_BUSY ){
2189 return rc;
drh73b64e42010-05-30 19:55:15 +00002190 }
2191 }
2192 }
drh658d76c2011-02-19 15:22:14 +00002193 if( mxI==0 ){
drh5bf39342011-06-02 17:24:49 +00002194 assert( rc==SQLITE_BUSY || (pWal->readOnly & WAL_SHM_RDONLY)!=0 );
dan4edc6bf2011-05-10 17:31:29 +00002195 return rc==SQLITE_BUSY ? WAL_RETRY : SQLITE_READONLY_CANTLOCK;
drh658d76c2011-02-19 15:22:14 +00002196 }
drh73b64e42010-05-30 19:55:15 +00002197
2198 rc = walLockShared(pWal, WAL_READ_LOCK(mxI));
2199 if( rc ){
2200 return rc==SQLITE_BUSY ? WAL_RETRY : rc;
2201 }
daneb8cb3a2010-06-05 18:34:26 +00002202 /* Now that the read-lock has been obtained, check that neither the
2203 ** value in the aReadMark[] array or the contents of the wal-index
2204 ** header have changed.
2205 **
2206 ** It is necessary to check that the wal-index header did not change
2207 ** between the time it was read and when the shared-lock was obtained
2208 ** on WAL_READ_LOCK(mxI) was obtained to account for the possibility
2209 ** that the log file may have been wrapped by a writer, or that frames
2210 ** that occur later in the log than pWal->hdr.mxFrame may have been
2211 ** copied into the database by a checkpointer. If either of these things
2212 ** happened, then reading the database with the current value of
2213 ** pWal->hdr.mxFrame risks reading a corrupted snapshot. So, retry
2214 ** instead.
2215 **
dan640aac42010-06-05 19:18:59 +00002216 ** This does not guarantee that the copy of the wal-index header is up to
2217 ** date before proceeding. That would not be possible without somehow
2218 ** blocking writers. It only guarantees that a dangerous checkpoint or
daneb8cb3a2010-06-05 18:34:26 +00002219 ** log-wrap (either of which would require an exclusive lock on
2220 ** WAL_READ_LOCK(mxI)) has not occurred since the snapshot was valid.
2221 */
dan8c408002010-11-01 17:38:24 +00002222 walShmBarrier(pWal);
drh73b64e42010-05-30 19:55:15 +00002223 if( pInfo->aReadMark[mxI]!=mxReadMark
dan4280eb32010-06-12 12:02:35 +00002224 || memcmp((void *)walIndexHdr(pWal), &pWal->hdr, sizeof(WalIndexHdr))
drh73b64e42010-05-30 19:55:15 +00002225 ){
2226 walUnlockShared(pWal, WAL_READ_LOCK(mxI));
2227 return WAL_RETRY;
2228 }else{
drhdb7f6472010-06-09 14:45:12 +00002229 assert( mxReadMark<=pWal->hdr.mxFrame );
shaneh5eba1f62010-07-02 17:05:03 +00002230 pWal->readLock = (i16)mxI;
drh73b64e42010-05-30 19:55:15 +00002231 }
2232 }
2233 return rc;
2234}
2235
2236/*
2237** Begin a read transaction on the database.
2238**
2239** This routine used to be called sqlite3OpenSnapshot() and with good reason:
2240** it takes a snapshot of the state of the WAL and wal-index for the current
2241** instant in time. The current thread will continue to use this snapshot.
2242** Other threads might append new content to the WAL and wal-index but
2243** that extra content is ignored by the current thread.
2244**
2245** If the database contents have changes since the previous read
2246** transaction, then *pChanged is set to 1 before returning. The
2247** Pager layer will use this to know that is cache is stale and
2248** needs to be flushed.
2249*/
drh66dfec8b2011-06-01 20:01:49 +00002250int sqlite3WalBeginReadTransaction(Wal *pWal, int *pChanged){
drh73b64e42010-05-30 19:55:15 +00002251 int rc; /* Return code */
drhaab4c022010-06-02 14:45:51 +00002252 int cnt = 0; /* Number of TryBeginRead attempts */
drh73b64e42010-05-30 19:55:15 +00002253
2254 do{
drhaab4c022010-06-02 14:45:51 +00002255 rc = walTryBeginRead(pWal, pChanged, 0, ++cnt);
drh73b64e42010-05-30 19:55:15 +00002256 }while( rc==WAL_RETRY );
drhab1cc742011-02-19 16:51:45 +00002257 testcase( (rc&0xff)==SQLITE_BUSY );
2258 testcase( (rc&0xff)==SQLITE_IOERR );
2259 testcase( rc==SQLITE_PROTOCOL );
2260 testcase( rc==SQLITE_OK );
dan7c246102010-04-12 19:00:29 +00002261 return rc;
2262}
2263
2264/*
drh73b64e42010-05-30 19:55:15 +00002265** Finish with a read transaction. All this does is release the
2266** read-lock.
dan7c246102010-04-12 19:00:29 +00002267*/
drh73b64e42010-05-30 19:55:15 +00002268void sqlite3WalEndReadTransaction(Wal *pWal){
dan73d66fd2010-08-07 16:17:48 +00002269 sqlite3WalEndWriteTransaction(pWal);
drh73b64e42010-05-30 19:55:15 +00002270 if( pWal->readLock>=0 ){
2271 walUnlockShared(pWal, WAL_READ_LOCK(pWal->readLock));
2272 pWal->readLock = -1;
2273 }
dan7c246102010-04-12 19:00:29 +00002274}
2275
dan5e0ce872010-04-28 17:48:44 +00002276/*
drh73b64e42010-05-30 19:55:15 +00002277** Read a page from the WAL, if it is present in the WAL and if the
2278** current read transaction is configured to use the WAL.
2279**
2280** The *pInWal is set to 1 if the requested page is in the WAL and
2281** has been loaded. Or *pInWal is set to 0 if the page was not in
2282** the WAL and needs to be read out of the database.
dan7c246102010-04-12 19:00:29 +00002283*/
danb6e099a2010-05-04 14:47:39 +00002284int sqlite3WalRead(
danbb23aff2010-05-10 14:46:09 +00002285 Wal *pWal, /* WAL handle */
2286 Pgno pgno, /* Database page number to read data for */
2287 int *pInWal, /* OUT: True if data is read from WAL */
2288 int nOut, /* Size of buffer pOut in bytes */
2289 u8 *pOut /* Buffer to write page data to */
danb6e099a2010-05-04 14:47:39 +00002290){
danbb23aff2010-05-10 14:46:09 +00002291 u32 iRead = 0; /* If !=0, WAL frame to return data from */
drh027a1282010-05-19 01:53:53 +00002292 u32 iLast = pWal->hdr.mxFrame; /* Last page in WAL for this reader */
danbb23aff2010-05-10 14:46:09 +00002293 int iHash; /* Used to loop through N hash tables */
dan7c246102010-04-12 19:00:29 +00002294
drhaab4c022010-06-02 14:45:51 +00002295 /* This routine is only be called from within a read transaction. */
2296 assert( pWal->readLock>=0 || pWal->lockError );
drh73b64e42010-05-30 19:55:15 +00002297
danbb23aff2010-05-10 14:46:09 +00002298 /* If the "last page" field of the wal-index header snapshot is 0, then
2299 ** no data will be read from the wal under any circumstances. Return early
drha927e942010-06-24 02:46:48 +00002300 ** in this case as an optimization. Likewise, if pWal->readLock==0,
2301 ** then the WAL is ignored by the reader so return early, as if the
2302 ** WAL were empty.
danbb23aff2010-05-10 14:46:09 +00002303 */
drh73b64e42010-05-30 19:55:15 +00002304 if( iLast==0 || pWal->readLock==0 ){
danbb23aff2010-05-10 14:46:09 +00002305 *pInWal = 0;
2306 return SQLITE_OK;
2307 }
2308
danbb23aff2010-05-10 14:46:09 +00002309 /* Search the hash table or tables for an entry matching page number
2310 ** pgno. Each iteration of the following for() loop searches one
2311 ** hash table (each hash table indexes up to HASHTABLE_NPAGE frames).
2312 **
drha927e942010-06-24 02:46:48 +00002313 ** This code might run concurrently to the code in walIndexAppend()
danbb23aff2010-05-10 14:46:09 +00002314 ** that adds entries to the wal-index (and possibly to this hash
drh6e810962010-05-19 17:49:50 +00002315 ** table). This means the value just read from the hash
danbb23aff2010-05-10 14:46:09 +00002316 ** slot (aHash[iKey]) may have been added before or after the
2317 ** current read transaction was opened. Values added after the
2318 ** read transaction was opened may have been written incorrectly -
2319 ** i.e. these slots may contain garbage data. However, we assume
2320 ** that any slots written before the current read transaction was
2321 ** opened remain unmodified.
2322 **
2323 ** For the reasons above, the if(...) condition featured in the inner
2324 ** loop of the following block is more stringent that would be required
2325 ** if we had exclusive access to the hash-table:
2326 **
2327 ** (aPgno[iFrame]==pgno):
2328 ** This condition filters out normal hash-table collisions.
2329 **
2330 ** (iFrame<=iLast):
2331 ** This condition filters out entries that were added to the hash
2332 ** table after the current read-transaction had started.
dan7c246102010-04-12 19:00:29 +00002333 */
dan13a3cb82010-06-11 19:04:21 +00002334 for(iHash=walFramePage(iLast); iHash>=0 && iRead==0; iHash--){
dan067f3162010-06-14 10:30:12 +00002335 volatile ht_slot *aHash; /* Pointer to hash table */
2336 volatile u32 *aPgno; /* Pointer to array of page numbers */
danbb23aff2010-05-10 14:46:09 +00002337 u32 iZero; /* Frame number corresponding to aPgno[0] */
2338 int iKey; /* Hash slot index */
drh519426a2010-07-09 03:19:07 +00002339 int nCollide; /* Number of hash collisions remaining */
2340 int rc; /* Error code */
danbb23aff2010-05-10 14:46:09 +00002341
dan4280eb32010-06-12 12:02:35 +00002342 rc = walHashGet(pWal, iHash, &aHash, &aPgno, &iZero);
2343 if( rc!=SQLITE_OK ){
2344 return rc;
2345 }
drh519426a2010-07-09 03:19:07 +00002346 nCollide = HASHTABLE_NSLOT;
dan6f150142010-05-21 15:31:56 +00002347 for(iKey=walHash(pgno); aHash[iKey]; iKey=walNextHash(iKey)){
danbb23aff2010-05-10 14:46:09 +00002348 u32 iFrame = aHash[iKey] + iZero;
dand60bf112010-06-14 11:18:50 +00002349 if( iFrame<=iLast && aPgno[aHash[iKey]]==pgno ){
drhd5156602011-11-12 16:46:55 +00002350 /* assert( iFrame>iRead ); -- not true if there is corruption */
danbb23aff2010-05-10 14:46:09 +00002351 iRead = iFrame;
2352 }
drh519426a2010-07-09 03:19:07 +00002353 if( (nCollide--)==0 ){
2354 return SQLITE_CORRUPT_BKPT;
2355 }
dan7c246102010-04-12 19:00:29 +00002356 }
2357 }
dan7c246102010-04-12 19:00:29 +00002358
danbb23aff2010-05-10 14:46:09 +00002359#ifdef SQLITE_ENABLE_EXPENSIVE_ASSERT
2360 /* If expensive assert() statements are available, do a linear search
2361 ** of the wal-index file content. Make sure the results agree with the
2362 ** result obtained using the hash indexes above. */
2363 {
2364 u32 iRead2 = 0;
2365 u32 iTest;
2366 for(iTest=iLast; iTest>0; iTest--){
dan13a3cb82010-06-11 19:04:21 +00002367 if( walFramePgno(pWal, iTest)==pgno ){
danbb23aff2010-05-10 14:46:09 +00002368 iRead2 = iTest;
dan7c246102010-04-12 19:00:29 +00002369 break;
2370 }
dan7c246102010-04-12 19:00:29 +00002371 }
danbb23aff2010-05-10 14:46:09 +00002372 assert( iRead==iRead2 );
dan7c246102010-04-12 19:00:29 +00002373 }
danbb23aff2010-05-10 14:46:09 +00002374#endif
dancd11fb22010-04-26 10:40:52 +00002375
dan7c246102010-04-12 19:00:29 +00002376 /* If iRead is non-zero, then it is the log frame number that contains the
2377 ** required page. Read and return data from the log file.
2378 */
2379 if( iRead ){
drhb2eced52010-08-12 02:41:12 +00002380 int sz;
2381 i64 iOffset;
2382 sz = pWal->hdr.szPage;
drhb07028f2011-10-14 21:49:18 +00002383 sz = (sz&0xfe00) + ((sz&0x0001)<<16);
drh9b78f792010-08-14 21:21:24 +00002384 testcase( sz<=32768 );
2385 testcase( sz>=65536 );
drhb2eced52010-08-12 02:41:12 +00002386 iOffset = walFrameOffset(iRead, sz) + WAL_FRAME_HDRSIZE;
drh7ed91f22010-04-29 22:34:07 +00002387 *pInWal = 1;
drh09b5dbc2010-07-07 14:35:58 +00002388 /* testcase( IS_BIG_INT(iOffset) ); // requires a 4GiB WAL */
drhd9e5c4f2010-05-12 18:01:39 +00002389 return sqlite3OsRead(pWal->pWalFd, pOut, nOut, iOffset);
dan7c246102010-04-12 19:00:29 +00002390 }
2391
drh7ed91f22010-04-29 22:34:07 +00002392 *pInWal = 0;
dan7c246102010-04-12 19:00:29 +00002393 return SQLITE_OK;
2394}
2395
2396
2397/*
dan763afe62010-08-03 06:42:39 +00002398** Return the size of the database in pages (or zero, if unknown).
dan7c246102010-04-12 19:00:29 +00002399*/
dan763afe62010-08-03 06:42:39 +00002400Pgno sqlite3WalDbsize(Wal *pWal){
drh7e9e70b2010-08-16 14:17:59 +00002401 if( pWal && ALWAYS(pWal->readLock>=0) ){
dan763afe62010-08-03 06:42:39 +00002402 return pWal->hdr.nPage;
2403 }
2404 return 0;
dan7c246102010-04-12 19:00:29 +00002405}
2406
dan30c86292010-04-30 16:24:46 +00002407
drh73b64e42010-05-30 19:55:15 +00002408/*
2409** This function starts a write transaction on the WAL.
2410**
2411** A read transaction must have already been started by a prior call
2412** to sqlite3WalBeginReadTransaction().
2413**
2414** If another thread or process has written into the database since
2415** the read transaction was started, then it is not possible for this
2416** thread to write as doing so would cause a fork. So this routine
2417** returns SQLITE_BUSY in that case and no write transaction is started.
2418**
2419** There can only be a single writer active at a time.
2420*/
2421int sqlite3WalBeginWriteTransaction(Wal *pWal){
2422 int rc;
drh73b64e42010-05-30 19:55:15 +00002423
2424 /* Cannot start a write transaction without first holding a read
2425 ** transaction. */
2426 assert( pWal->readLock>=0 );
2427
dan1e5de5a2010-07-15 18:20:53 +00002428 if( pWal->readOnly ){
2429 return SQLITE_READONLY;
2430 }
2431
drh73b64e42010-05-30 19:55:15 +00002432 /* Only one writer allowed at a time. Get the write lock. Return
2433 ** SQLITE_BUSY if unable.
2434 */
2435 rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1);
2436 if( rc ){
2437 return rc;
2438 }
drhc99597c2010-05-31 01:41:15 +00002439 pWal->writeLock = 1;
drh73b64e42010-05-30 19:55:15 +00002440
2441 /* If another connection has written to the database file since the
2442 ** time the read transaction on this connection was started, then
2443 ** the write is disallowed.
2444 */
dan4280eb32010-06-12 12:02:35 +00002445 if( memcmp(&pWal->hdr, (void *)walIndexHdr(pWal), sizeof(WalIndexHdr))!=0 ){
drh73b64e42010-05-30 19:55:15 +00002446 walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1);
drhc99597c2010-05-31 01:41:15 +00002447 pWal->writeLock = 0;
dan9971e712010-06-01 15:44:57 +00002448 rc = SQLITE_BUSY;
drh73b64e42010-05-30 19:55:15 +00002449 }
2450
drh7ed91f22010-04-29 22:34:07 +00002451 return rc;
dan7c246102010-04-12 19:00:29 +00002452}
2453
dan74d6cd82010-04-24 18:44:05 +00002454/*
drh73b64e42010-05-30 19:55:15 +00002455** End a write transaction. The commit has already been done. This
2456** routine merely releases the lock.
2457*/
2458int sqlite3WalEndWriteTransaction(Wal *pWal){
danda9fe0c2010-07-13 18:44:03 +00002459 if( pWal->writeLock ){
2460 walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1);
2461 pWal->writeLock = 0;
danf60b7f32011-12-16 13:24:27 +00002462 pWal->truncateOnCommit = 0;
danda9fe0c2010-07-13 18:44:03 +00002463 }
drh73b64e42010-05-30 19:55:15 +00002464 return SQLITE_OK;
2465}
2466
2467/*
dan74d6cd82010-04-24 18:44:05 +00002468** If any data has been written (but not committed) to the log file, this
2469** function moves the write-pointer back to the start of the transaction.
2470**
2471** Additionally, the callback function is invoked for each frame written
drh73b64e42010-05-30 19:55:15 +00002472** to the WAL since the start of the transaction. If the callback returns
dan74d6cd82010-04-24 18:44:05 +00002473** other than SQLITE_OK, it is not invoked again and the error code is
2474** returned to the caller.
2475**
2476** Otherwise, if the callback function does not return an error, this
2477** function returns SQLITE_OK.
2478*/
drh7ed91f22010-04-29 22:34:07 +00002479int sqlite3WalUndo(Wal *pWal, int (*xUndo)(void *, Pgno), void *pUndoCtx){
dan55437592010-05-11 12:19:26 +00002480 int rc = SQLITE_OK;
drh7e9e70b2010-08-16 14:17:59 +00002481 if( ALWAYS(pWal->writeLock) ){
drh027a1282010-05-19 01:53:53 +00002482 Pgno iMax = pWal->hdr.mxFrame;
dan55437592010-05-11 12:19:26 +00002483 Pgno iFrame;
2484
dan5d656852010-06-14 07:53:26 +00002485 /* Restore the clients cache of the wal-index header to the state it
2486 ** was in before the client began writing to the database.
2487 */
dan067f3162010-06-14 10:30:12 +00002488 memcpy(&pWal->hdr, (void *)walIndexHdr(pWal), sizeof(WalIndexHdr));
dan5d656852010-06-14 07:53:26 +00002489
2490 for(iFrame=pWal->hdr.mxFrame+1;
2491 ALWAYS(rc==SQLITE_OK) && iFrame<=iMax;
2492 iFrame++
2493 ){
2494 /* This call cannot fail. Unless the page for which the page number
2495 ** is passed as the second argument is (a) in the cache and
2496 ** (b) has an outstanding reference, then xUndo is either a no-op
2497 ** (if (a) is false) or simply expels the page from the cache (if (b)
2498 ** is false).
2499 **
2500 ** If the upper layer is doing a rollback, it is guaranteed that there
2501 ** are no outstanding references to any page other than page 1. And
2502 ** page 1 is never written to the log until the transaction is
2503 ** committed. As a result, the call to xUndo may not fail.
2504 */
dan5d656852010-06-14 07:53:26 +00002505 assert( walFramePgno(pWal, iFrame)!=1 );
2506 rc = xUndo(pUndoCtx, walFramePgno(pWal, iFrame));
dan6f150142010-05-21 15:31:56 +00002507 }
dan5d656852010-06-14 07:53:26 +00002508 walCleanupHash(pWal);
dan74d6cd82010-04-24 18:44:05 +00002509 }
dan5d656852010-06-14 07:53:26 +00002510 assert( rc==SQLITE_OK );
dan74d6cd82010-04-24 18:44:05 +00002511 return rc;
2512}
2513
dan71d89912010-05-24 13:57:42 +00002514/*
2515** Argument aWalData must point to an array of WAL_SAVEPOINT_NDATA u32
2516** values. This function populates the array with values required to
2517** "rollback" the write position of the WAL handle back to the current
2518** point in the event of a savepoint rollback (via WalSavepointUndo()).
drh7ed91f22010-04-29 22:34:07 +00002519*/
dan71d89912010-05-24 13:57:42 +00002520void sqlite3WalSavepoint(Wal *pWal, u32 *aWalData){
drh73b64e42010-05-30 19:55:15 +00002521 assert( pWal->writeLock );
dan71d89912010-05-24 13:57:42 +00002522 aWalData[0] = pWal->hdr.mxFrame;
2523 aWalData[1] = pWal->hdr.aFrameCksum[0];
2524 aWalData[2] = pWal->hdr.aFrameCksum[1];
dan6e6bd562010-06-02 18:59:03 +00002525 aWalData[3] = pWal->nCkpt;
dan4cd78b42010-04-26 16:57:10 +00002526}
2527
dan71d89912010-05-24 13:57:42 +00002528/*
2529** Move the write position of the WAL back to the point identified by
2530** the values in the aWalData[] array. aWalData must point to an array
2531** of WAL_SAVEPOINT_NDATA u32 values that has been previously populated
2532** by a call to WalSavepoint().
drh7ed91f22010-04-29 22:34:07 +00002533*/
dan71d89912010-05-24 13:57:42 +00002534int sqlite3WalSavepointUndo(Wal *pWal, u32 *aWalData){
dan4cd78b42010-04-26 16:57:10 +00002535 int rc = SQLITE_OK;
dan4cd78b42010-04-26 16:57:10 +00002536
dan6e6bd562010-06-02 18:59:03 +00002537 assert( pWal->writeLock );
2538 assert( aWalData[3]!=pWal->nCkpt || aWalData[0]<=pWal->hdr.mxFrame );
2539
2540 if( aWalData[3]!=pWal->nCkpt ){
2541 /* This savepoint was opened immediately after the write-transaction
2542 ** was started. Right after that, the writer decided to wrap around
2543 ** to the start of the log. Update the savepoint values to match.
2544 */
2545 aWalData[0] = 0;
2546 aWalData[3] = pWal->nCkpt;
2547 }
2548
dan71d89912010-05-24 13:57:42 +00002549 if( aWalData[0]<pWal->hdr.mxFrame ){
dan71d89912010-05-24 13:57:42 +00002550 pWal->hdr.mxFrame = aWalData[0];
2551 pWal->hdr.aFrameCksum[0] = aWalData[1];
2552 pWal->hdr.aFrameCksum[1] = aWalData[2];
dan5d656852010-06-14 07:53:26 +00002553 walCleanupHash(pWal);
dan6f150142010-05-21 15:31:56 +00002554 }
dan6e6bd562010-06-02 18:59:03 +00002555
dan4cd78b42010-04-26 16:57:10 +00002556 return rc;
2557}
2558
drh8dd4afa2011-12-08 19:50:32 +00002559
dan9971e712010-06-01 15:44:57 +00002560/*
2561** This function is called just before writing a set of frames to the log
2562** file (see sqlite3WalFrames()). It checks to see if, instead of appending
2563** to the current log file, it is possible to overwrite the start of the
2564** existing log file with the new frames (i.e. "reset" the log). If so,
2565** it sets pWal->hdr.mxFrame to 0. Otherwise, pWal->hdr.mxFrame is left
2566** unchanged.
2567**
2568** SQLITE_OK is returned if no error is encountered (regardless of whether
2569** or not pWal->hdr.mxFrame is modified). An SQLite error code is returned
drh4533cd02010-10-05 15:41:05 +00002570** if an error occurs.
dan9971e712010-06-01 15:44:57 +00002571*/
2572static int walRestartLog(Wal *pWal){
2573 int rc = SQLITE_OK;
drhaab4c022010-06-02 14:45:51 +00002574 int cnt;
2575
dan13a3cb82010-06-11 19:04:21 +00002576 if( pWal->readLock==0 ){
dan9971e712010-06-01 15:44:57 +00002577 volatile WalCkptInfo *pInfo = walCkptInfo(pWal);
2578 assert( pInfo->nBackfill==pWal->hdr.mxFrame );
2579 if( pInfo->nBackfill>0 ){
drh658d76c2011-02-19 15:22:14 +00002580 u32 salt1;
2581 sqlite3_randomness(4, &salt1);
dan9971e712010-06-01 15:44:57 +00002582 rc = walLockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);
2583 if( rc==SQLITE_OK ){
2584 /* If all readers are using WAL_READ_LOCK(0) (in other words if no
2585 ** readers are currently using the WAL), then the transactions
2586 ** frames will overwrite the start of the existing log. Update the
2587 ** wal-index header to reflect this.
2588 **
2589 ** In theory it would be Ok to update the cache of the header only
2590 ** at this point. But updating the actual wal-index header is also
2591 ** safe and means there is no special case for sqlite3WalUndo()
2592 ** to handle if this transaction is rolled back.
2593 */
dan199100e2010-06-09 16:58:49 +00002594 int i; /* Loop counter */
dan9971e712010-06-01 15:44:57 +00002595 u32 *aSalt = pWal->hdr.aSalt; /* Big-endian salt values */
drh85a83752011-05-16 21:00:27 +00002596
dan9971e712010-06-01 15:44:57 +00002597 pWal->nCkpt++;
2598 pWal->hdr.mxFrame = 0;
2599 sqlite3Put4byte((u8*)&aSalt[0], 1 + sqlite3Get4byte((u8*)&aSalt[0]));
drh658d76c2011-02-19 15:22:14 +00002600 aSalt[1] = salt1;
dan9971e712010-06-01 15:44:57 +00002601 walIndexWriteHdr(pWal);
dan199100e2010-06-09 16:58:49 +00002602 pInfo->nBackfill = 0;
2603 for(i=1; i<WAL_NREADER; i++) pInfo->aReadMark[i] = READMARK_NOT_USED;
2604 assert( pInfo->aReadMark[0]==0 );
dan9971e712010-06-01 15:44:57 +00002605 walUnlockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);
drh4533cd02010-10-05 15:41:05 +00002606 }else if( rc!=SQLITE_BUSY ){
2607 return rc;
dan9971e712010-06-01 15:44:57 +00002608 }
2609 }
2610 walUnlockShared(pWal, WAL_READ_LOCK(0));
2611 pWal->readLock = -1;
drhaab4c022010-06-02 14:45:51 +00002612 cnt = 0;
dan9971e712010-06-01 15:44:57 +00002613 do{
2614 int notUsed;
drhaab4c022010-06-02 14:45:51 +00002615 rc = walTryBeginRead(pWal, &notUsed, 1, ++cnt);
dan9971e712010-06-01 15:44:57 +00002616 }while( rc==WAL_RETRY );
drhc90e0812011-02-19 17:02:44 +00002617 assert( (rc&0xff)!=SQLITE_BUSY ); /* BUSY not possible when useWal==1 */
drhab1cc742011-02-19 16:51:45 +00002618 testcase( (rc&0xff)==SQLITE_IOERR );
2619 testcase( rc==SQLITE_PROTOCOL );
2620 testcase( rc==SQLITE_OK );
dan9971e712010-06-01 15:44:57 +00002621 }
2622 return rc;
2623}
2624
drh88f975a2011-12-16 19:34:36 +00002625/*
2626** Write iAmt bytes of content into the WAL file beginning at iOffset.
2627**
2628** When crossing the boundary between the first and second sectors of the
2629** file, first write all of the first sector content, then fsync(), then
2630** continue writing content for the second sector. This ensures that
2631** the WAL header is overwritten before the first commit mark.
2632*/
2633static int walWriteToLog(
2634 Wal *pWal, /* WAL to write to */
2635 void *pContent, /* Content to be written */
2636 int iAmt, /* Number of bytes to write */
2637 sqlite3_int64 iOffset /* Start writing at this offset */
2638){
2639 int rc;
drh4eb02a42011-12-16 21:26:26 +00002640 if( iOffset>=pWal->szFirstBlock
2641 || iOffset+iAmt<pWal->szFirstBlock
2642 || pWal->syncFlags==0
2643 ){
drh88f975a2011-12-16 19:34:36 +00002644 /* The common and fast case. Just write the data. */
2645 rc = sqlite3OsWrite(pWal->pWalFd, pContent, iAmt, iOffset);
2646 }else{
2647 /* If this write will cross the first sector boundary, it has to
2648 ** be split it two with a sync in between. */
2649 int iFirstAmt = pWal->szFirstBlock - iOffset;
2650 assert( iFirstAmt>0 && iFirstAmt<iAmt );
2651 rc = sqlite3OsWrite(pWal->pWalFd, pContent, iFirstAmt, iOffset);
2652 if( rc ) return rc;
drh4eb02a42011-12-16 21:26:26 +00002653 assert( pWal->syncFlags & (SQLITE_SYNC_NORMAL|SQLITE_SYNC_FULL) );
2654 rc = sqlite3OsSync(pWal->pWalFd, pWal->syncFlags);
drh88f975a2011-12-16 19:34:36 +00002655 if( rc ) return rc;
2656 pContent = (void*)(iFirstAmt + (char*)pContent);
2657 rc = sqlite3OsWrite(pWal->pWalFd, pContent,
2658 iAmt-iFirstAmt, iOffset+iFirstAmt);
2659 }
2660 return rc;
2661}
2662
dan7c246102010-04-12 19:00:29 +00002663/*
dan4cd78b42010-04-26 16:57:10 +00002664** Write a set of frames to the log. The caller must hold the write-lock
dan9971e712010-06-01 15:44:57 +00002665** on the log file (obtained using sqlite3WalBeginWriteTransaction()).
dan7c246102010-04-12 19:00:29 +00002666*/
drhc438efd2010-04-26 00:19:45 +00002667int sqlite3WalFrames(
drh7ed91f22010-04-29 22:34:07 +00002668 Wal *pWal, /* Wal handle to write to */
drh6e810962010-05-19 17:49:50 +00002669 int szPage, /* Database page-size in bytes */
dan7c246102010-04-12 19:00:29 +00002670 PgHdr *pList, /* List of dirty pages to write */
2671 Pgno nTruncate, /* Database size after this commit */
2672 int isCommit, /* True if this is a commit */
danc5118782010-04-17 17:34:41 +00002673 int sync_flags /* Flags to pass to OsSync() (or 0) */
dan7c246102010-04-12 19:00:29 +00002674){
dan7c246102010-04-12 19:00:29 +00002675 int rc; /* Used to catch return codes */
2676 u32 iFrame; /* Next frame address */
drh7ed91f22010-04-29 22:34:07 +00002677 u8 aFrame[WAL_FRAME_HDRSIZE]; /* Buffer to assemble frame-header in */
dan7c246102010-04-12 19:00:29 +00002678 PgHdr *p; /* Iterator to run through pList with. */
drhe874d9e2010-05-07 20:02:23 +00002679 PgHdr *pLast = 0; /* Last frame in list */
dan7c246102010-04-12 19:00:29 +00002680 int nLast = 0; /* Number of extra copies of last page */
2681
dan7c246102010-04-12 19:00:29 +00002682 assert( pList );
drh73b64e42010-05-30 19:55:15 +00002683 assert( pWal->writeLock );
dan7c246102010-04-12 19:00:29 +00002684
drhc74c3332010-05-31 12:15:19 +00002685#if defined(SQLITE_TEST) && defined(SQLITE_DEBUG)
2686 { int cnt; for(cnt=0, p=pList; p; p=p->pDirty, cnt++){}
2687 WALTRACE(("WAL%p: frame write begin. %d frames. mxFrame=%d. %s\n",
2688 pWal, cnt, pWal->hdr.mxFrame, isCommit ? "Commit" : "Spill"));
2689 }
2690#endif
2691
dan9971e712010-06-01 15:44:57 +00002692 /* See if it is possible to write these frames into the start of the
2693 ** log file, instead of appending to it at pWal->hdr.mxFrame.
2694 */
2695 if( SQLITE_OK!=(rc = walRestartLog(pWal)) ){
dan9971e712010-06-01 15:44:57 +00002696 return rc;
2697 }
dan9971e712010-06-01 15:44:57 +00002698
drha2a42012010-05-18 18:01:08 +00002699 /* If this is the first frame written into the log, write the WAL
2700 ** header to the start of the WAL file. See comments at the top of
2701 ** this source file for a description of the WAL header format.
dan97a31352010-04-16 13:59:31 +00002702 */
drh027a1282010-05-19 01:53:53 +00002703 iFrame = pWal->hdr.mxFrame;
dan97a31352010-04-16 13:59:31 +00002704 if( iFrame==0 ){
dan10f5a502010-06-23 15:55:43 +00002705 u8 aWalHdr[WAL_HDRSIZE]; /* Buffer to assemble wal-header in */
2706 u32 aCksum[2]; /* Checksum for wal-header */
2707
danb8fd6c22010-05-24 10:39:36 +00002708 sqlite3Put4byte(&aWalHdr[0], (WAL_MAGIC | SQLITE_BIGENDIAN));
dan10f5a502010-06-23 15:55:43 +00002709 sqlite3Put4byte(&aWalHdr[4], WAL_MAX_VERSION);
drh23ea97b2010-05-20 16:45:58 +00002710 sqlite3Put4byte(&aWalHdr[8], szPage);
2711 sqlite3Put4byte(&aWalHdr[12], pWal->nCkpt);
drh2327f5a2010-07-07 21:06:48 +00002712 sqlite3_randomness(8, pWal->hdr.aSalt);
drh7e263722010-05-20 21:21:09 +00002713 memcpy(&aWalHdr[16], pWal->hdr.aSalt, 8);
dan10f5a502010-06-23 15:55:43 +00002714 walChecksumBytes(1, aWalHdr, WAL_HDRSIZE-2*4, 0, aCksum);
2715 sqlite3Put4byte(&aWalHdr[24], aCksum[0]);
2716 sqlite3Put4byte(&aWalHdr[28], aCksum[1]);
2717
drhb2eced52010-08-12 02:41:12 +00002718 pWal->szPage = szPage;
dan10f5a502010-06-23 15:55:43 +00002719 pWal->hdr.bigEndCksum = SQLITE_BIGENDIAN;
2720 pWal->hdr.aFrameCksum[0] = aCksum[0];
2721 pWal->hdr.aFrameCksum[1] = aCksum[1];
danf60b7f32011-12-16 13:24:27 +00002722 pWal->truncateOnCommit = 1;
dan10f5a502010-06-23 15:55:43 +00002723
drh23ea97b2010-05-20 16:45:58 +00002724 rc = sqlite3OsWrite(pWal->pWalFd, aWalHdr, sizeof(aWalHdr), 0);
drhc74c3332010-05-31 12:15:19 +00002725 WALTRACE(("WAL%p: wal-header write %s\n", pWal, rc ? "failed" : "ok"));
dan97a31352010-04-16 13:59:31 +00002726 if( rc!=SQLITE_OK ){
2727 return rc;
2728 }
2729 }
shanehbd2aaf92010-09-01 02:38:21 +00002730 assert( (int)pWal->szPage==szPage );
dan97a31352010-04-16 13:59:31 +00002731
drh4eb02a42011-12-16 21:26:26 +00002732 /* Setup information needed to do the WAL header sync */
2733 if( pWal->noSyncHeader ){
2734 assert( pWal->szFirstBlock==0 );
2735 assert( pWal->syncFlags==0 );
2736 }else{
2737 pWal->szFirstBlock = sqlite3OsSectorSize(pWal->pWalFd);
2738 if( szPage>pWal->szFirstBlock ) pWal->szFirstBlock = szPage;
2739 pWal->syncFlags = sync_flags & SQLITE_SYNC_MASK;
2740 }
drh88f975a2011-12-16 19:34:36 +00002741
dan9971e712010-06-01 15:44:57 +00002742 /* Write the log file. */
dan7c246102010-04-12 19:00:29 +00002743 for(p=pList; p; p=p->pDirty){
2744 u32 nDbsize; /* Db-size field for frame header */
2745 i64 iOffset; /* Write offset in log file */
dan47ee3862010-06-22 15:18:44 +00002746 void *pData;
2747
drh6e810962010-05-19 17:49:50 +00002748 iOffset = walFrameOffset(++iFrame, szPage);
drhe9187b42010-07-07 14:39:59 +00002749 /* testcase( IS_BIG_INT(iOffset) ); // requires a 4GiB WAL */
dan7c246102010-04-12 19:00:29 +00002750
2751 /* Populate and write the frame header */
2752 nDbsize = (isCommit && p->pDirty==0) ? nTruncate : 0;
drha7152112010-06-22 21:15:49 +00002753#if defined(SQLITE_HAS_CODEC)
dan47ee3862010-06-22 15:18:44 +00002754 if( (pData = sqlite3PagerCodec(p))==0 ) return SQLITE_NOMEM;
drha7152112010-06-22 21:15:49 +00002755#else
2756 pData = p->pData;
2757#endif
dan47ee3862010-06-22 15:18:44 +00002758 walEncodeFrame(pWal, p->pgno, nDbsize, pData, aFrame);
drh88f975a2011-12-16 19:34:36 +00002759 rc = walWriteToLog(pWal, aFrame, sizeof(aFrame), iOffset);
dan7c246102010-04-12 19:00:29 +00002760 if( rc!=SQLITE_OK ){
2761 return rc;
2762 }
2763
2764 /* Write the page data */
drh88f975a2011-12-16 19:34:36 +00002765 rc = walWriteToLog(pWal, pData, szPage, iOffset+sizeof(aFrame));
dan7c246102010-04-12 19:00:29 +00002766 if( rc!=SQLITE_OK ){
2767 return rc;
2768 }
2769 pLast = p;
2770 }
2771
2772 /* Sync the log file if the 'isSync' flag was specified. */
drh4eb02a42011-12-16 21:26:26 +00002773 if( isCommit && (sync_flags & WAL_SYNC_TRANSACTIONS)!=0 ){
drhd9e5c4f2010-05-12 18:01:39 +00002774 i64 iSegment = sqlite3OsSectorSize(pWal->pWalFd);
drh6e810962010-05-19 17:49:50 +00002775 i64 iOffset = walFrameOffset(iFrame+1, szPage);
dan67032392010-04-17 15:42:43 +00002776
drh69c46962010-05-17 20:16:50 +00002777 assert( iSegment>0 );
dan7c246102010-04-12 19:00:29 +00002778
dan7c246102010-04-12 19:00:29 +00002779 iSegment = (((iOffset+iSegment-1)/iSegment) * iSegment);
2780 while( iOffset<iSegment ){
dan47ee3862010-06-22 15:18:44 +00002781 void *pData;
drha7152112010-06-22 21:15:49 +00002782#if defined(SQLITE_HAS_CODEC)
dan47ee3862010-06-22 15:18:44 +00002783 if( (pData = sqlite3PagerCodec(pLast))==0 ) return SQLITE_NOMEM;
drha7152112010-06-22 21:15:49 +00002784#else
2785 pData = pLast->pData;
2786#endif
dan47ee3862010-06-22 15:18:44 +00002787 walEncodeFrame(pWal, pLast->pgno, nTruncate, pData, aFrame);
drhe9187b42010-07-07 14:39:59 +00002788 /* testcase( IS_BIG_INT(iOffset) ); // requires a 4GiB WAL */
drh88f975a2011-12-16 19:34:36 +00002789 rc = walWriteToLog(pWal, aFrame, sizeof(aFrame), iOffset);
dan7c246102010-04-12 19:00:29 +00002790 if( rc!=SQLITE_OK ){
2791 return rc;
2792 }
drh7ed91f22010-04-29 22:34:07 +00002793 iOffset += WAL_FRAME_HDRSIZE;
drh88f975a2011-12-16 19:34:36 +00002794 rc = walWriteToLog(pWal, pData, szPage, iOffset);
dan7c246102010-04-12 19:00:29 +00002795 if( rc!=SQLITE_OK ){
2796 return rc;
2797 }
2798 nLast++;
drh6e810962010-05-19 17:49:50 +00002799 iOffset += szPage;
dan7c246102010-04-12 19:00:29 +00002800 }
dan7c246102010-04-12 19:00:29 +00002801
drh4eb02a42011-12-16 21:26:26 +00002802 rc = sqlite3OsSync(pWal->pWalFd, sync_flags & SQLITE_SYNC_MASK);
dan7c246102010-04-12 19:00:29 +00002803 }
2804
danf60b7f32011-12-16 13:24:27 +00002805 if( isCommit && pWal->truncateOnCommit && pWal->mxWalSize>=0 ){
2806 i64 sz = pWal->mxWalSize;
2807 if( walFrameOffset(iFrame+nLast+1, szPage)>pWal->mxWalSize ){
2808 sz = walFrameOffset(iFrame+nLast+1, szPage);
2809 }
2810 walLimitSize(pWal, sz);
2811 pWal->truncateOnCommit = 0;
2812 }
2813
drhe730fec2010-05-18 12:56:50 +00002814 /* Append data to the wal-index. It is not necessary to lock the
drha2a42012010-05-18 18:01:08 +00002815 ** wal-index to do this as the SQLITE_SHM_WRITE lock held on the wal-index
dan7c246102010-04-12 19:00:29 +00002816 ** guarantees that there are no other writers, and no data that may
2817 ** be in use by existing readers is being overwritten.
2818 */
drh027a1282010-05-19 01:53:53 +00002819 iFrame = pWal->hdr.mxFrame;
danc7991bd2010-05-05 19:04:59 +00002820 for(p=pList; p && rc==SQLITE_OK; p=p->pDirty){
dan7c246102010-04-12 19:00:29 +00002821 iFrame++;
danc7991bd2010-05-05 19:04:59 +00002822 rc = walIndexAppend(pWal, iFrame, p->pgno);
dan7c246102010-04-12 19:00:29 +00002823 }
danc7991bd2010-05-05 19:04:59 +00002824 while( nLast>0 && rc==SQLITE_OK ){
dan7c246102010-04-12 19:00:29 +00002825 iFrame++;
2826 nLast--;
danc7991bd2010-05-05 19:04:59 +00002827 rc = walIndexAppend(pWal, iFrame, pLast->pgno);
dan7c246102010-04-12 19:00:29 +00002828 }
2829
danc7991bd2010-05-05 19:04:59 +00002830 if( rc==SQLITE_OK ){
2831 /* Update the private copy of the header. */
shaneh1df2db72010-08-18 02:28:48 +00002832 pWal->hdr.szPage = (u16)((szPage&0xff00) | (szPage>>16));
drh9b78f792010-08-14 21:21:24 +00002833 testcase( szPage<=32768 );
2834 testcase( szPage>=65536 );
drh027a1282010-05-19 01:53:53 +00002835 pWal->hdr.mxFrame = iFrame;
danc7991bd2010-05-05 19:04:59 +00002836 if( isCommit ){
2837 pWal->hdr.iChange++;
2838 pWal->hdr.nPage = nTruncate;
2839 }
danc7991bd2010-05-05 19:04:59 +00002840 /* If this is a commit, update the wal-index header too. */
2841 if( isCommit ){
drh7e263722010-05-20 21:21:09 +00002842 walIndexWriteHdr(pWal);
danc7991bd2010-05-05 19:04:59 +00002843 pWal->iCallback = iFrame;
2844 }
dan7c246102010-04-12 19:00:29 +00002845 }
danc7991bd2010-05-05 19:04:59 +00002846
drhc74c3332010-05-31 12:15:19 +00002847 WALTRACE(("WAL%p: frame write %s\n", pWal, rc ? "failed" : "ok"));
dan8d22a172010-04-19 18:03:51 +00002848 return rc;
dan7c246102010-04-12 19:00:29 +00002849}
2850
2851/*
drh73b64e42010-05-30 19:55:15 +00002852** This routine is called to implement sqlite3_wal_checkpoint() and
2853** related interfaces.
danb9bf16b2010-04-14 11:23:30 +00002854**
drh73b64e42010-05-30 19:55:15 +00002855** Obtain a CHECKPOINT lock and then backfill as much information as
2856** we can from WAL into the database.
dana58f26f2010-11-16 18:56:51 +00002857**
2858** If parameter xBusy is not NULL, it is a pointer to a busy-handler
2859** callback. In this case this function runs a blocking checkpoint.
dan7c246102010-04-12 19:00:29 +00002860*/
drhc438efd2010-04-26 00:19:45 +00002861int sqlite3WalCheckpoint(
drh7ed91f22010-04-29 22:34:07 +00002862 Wal *pWal, /* Wal connection */
dancdc1f042010-11-18 12:11:05 +00002863 int eMode, /* PASSIVE, FULL or RESTART */
dana58f26f2010-11-16 18:56:51 +00002864 int (*xBusy)(void*), /* Function to call when busy */
2865 void *pBusyArg, /* Context argument for xBusyHandler */
danc5118782010-04-17 17:34:41 +00002866 int sync_flags, /* Flags to sync db file with (or 0) */
danb6e099a2010-05-04 14:47:39 +00002867 int nBuf, /* Size of temporary buffer */
dancdc1f042010-11-18 12:11:05 +00002868 u8 *zBuf, /* Temporary buffer to use */
2869 int *pnLog, /* OUT: Number of frames in WAL */
2870 int *pnCkpt /* OUT: Number of backfilled frames in WAL */
dan7c246102010-04-12 19:00:29 +00002871){
danb9bf16b2010-04-14 11:23:30 +00002872 int rc; /* Return code */
dan31c03902010-04-29 14:51:33 +00002873 int isChanged = 0; /* True if a new wal-index header is loaded */
danf2b8dd52010-11-18 19:28:01 +00002874 int eMode2 = eMode; /* Mode to pass to walCheckpoint() */
dan7c246102010-04-12 19:00:29 +00002875
dand54ff602010-05-31 11:16:30 +00002876 assert( pWal->ckptLock==0 );
dana58f26f2010-11-16 18:56:51 +00002877 assert( pWal->writeLock==0 );
dan39c79f52010-04-15 10:58:51 +00002878
drh66dfec8b2011-06-01 20:01:49 +00002879 if( pWal->readOnly ) return SQLITE_READONLY;
drhc74c3332010-05-31 12:15:19 +00002880 WALTRACE(("WAL%p: checkpoint begins\n", pWal));
drh73b64e42010-05-30 19:55:15 +00002881 rc = walLockExclusive(pWal, WAL_CKPT_LOCK, 1);
2882 if( rc ){
2883 /* Usually this is SQLITE_BUSY meaning that another thread or process
2884 ** is already running a checkpoint, or maybe a recovery. But it might
2885 ** also be SQLITE_IOERR. */
danb9bf16b2010-04-14 11:23:30 +00002886 return rc;
2887 }
dand54ff602010-05-31 11:16:30 +00002888 pWal->ckptLock = 1;
dan64d039e2010-04-13 19:27:31 +00002889
dana58f26f2010-11-16 18:56:51 +00002890 /* If this is a blocking-checkpoint, then obtain the write-lock as well
2891 ** to prevent any writers from running while the checkpoint is underway.
2892 ** This has to be done before the call to walIndexReadHdr() below.
danf2b8dd52010-11-18 19:28:01 +00002893 **
2894 ** If the writer lock cannot be obtained, then a passive checkpoint is
2895 ** run instead. Since the checkpointer is not holding the writer lock,
2896 ** there is no point in blocking waiting for any readers. Assuming no
2897 ** other error occurs, this function will return SQLITE_BUSY to the caller.
dana58f26f2010-11-16 18:56:51 +00002898 */
dancdc1f042010-11-18 12:11:05 +00002899 if( eMode!=SQLITE_CHECKPOINT_PASSIVE ){
dana58f26f2010-11-16 18:56:51 +00002900 rc = walBusyLock(pWal, xBusy, pBusyArg, WAL_WRITE_LOCK, 1);
danf2b8dd52010-11-18 19:28:01 +00002901 if( rc==SQLITE_OK ){
2902 pWal->writeLock = 1;
2903 }else if( rc==SQLITE_BUSY ){
2904 eMode2 = SQLITE_CHECKPOINT_PASSIVE;
2905 rc = SQLITE_OK;
2906 }
danb9bf16b2010-04-14 11:23:30 +00002907 }
dana58f26f2010-11-16 18:56:51 +00002908
danf2b8dd52010-11-18 19:28:01 +00002909 /* Read the wal-index header. */
drh7ed91f22010-04-29 22:34:07 +00002910 if( rc==SQLITE_OK ){
dana58f26f2010-11-16 18:56:51 +00002911 rc = walIndexReadHdr(pWal, &isChanged);
2912 }
danf2b8dd52010-11-18 19:28:01 +00002913
2914 /* Copy data from the log to the database file. */
dan9c5e3682011-02-07 15:12:12 +00002915 if( rc==SQLITE_OK ){
2916 if( pWal->hdr.mxFrame && walPagesize(pWal)!=nBuf ){
danf2b8dd52010-11-18 19:28:01 +00002917 rc = SQLITE_CORRUPT_BKPT;
2918 }else{
dan9c5e3682011-02-07 15:12:12 +00002919 rc = walCheckpoint(pWal, eMode2, xBusy, pBusyArg, sync_flags, zBuf);
2920 }
2921
2922 /* If no error occurred, set the output variables. */
2923 if( rc==SQLITE_OK || rc==SQLITE_BUSY ){
danf2b8dd52010-11-18 19:28:01 +00002924 if( pnLog ) *pnLog = (int)pWal->hdr.mxFrame;
dan9c5e3682011-02-07 15:12:12 +00002925 if( pnCkpt ) *pnCkpt = (int)(walCkptInfo(pWal)->nBackfill);
danf2b8dd52010-11-18 19:28:01 +00002926 }
danb9bf16b2010-04-14 11:23:30 +00002927 }
danf2b8dd52010-11-18 19:28:01 +00002928
dan31c03902010-04-29 14:51:33 +00002929 if( isChanged ){
2930 /* If a new wal-index header was loaded before the checkpoint was
drha2a42012010-05-18 18:01:08 +00002931 ** performed, then the pager-cache associated with pWal is now
dan31c03902010-04-29 14:51:33 +00002932 ** out of date. So zero the cached wal-index header to ensure that
2933 ** next time the pager opens a snapshot on this database it knows that
2934 ** the cache needs to be reset.
2935 */
2936 memset(&pWal->hdr, 0, sizeof(WalIndexHdr));
2937 }
danb9bf16b2010-04-14 11:23:30 +00002938
2939 /* Release the locks. */
dana58f26f2010-11-16 18:56:51 +00002940 sqlite3WalEndWriteTransaction(pWal);
drh73b64e42010-05-30 19:55:15 +00002941 walUnlockExclusive(pWal, WAL_CKPT_LOCK, 1);
dand54ff602010-05-31 11:16:30 +00002942 pWal->ckptLock = 0;
drhc74c3332010-05-31 12:15:19 +00002943 WALTRACE(("WAL%p: checkpoint %s\n", pWal, rc ? "failed" : "ok"));
danf2b8dd52010-11-18 19:28:01 +00002944 return (rc==SQLITE_OK && eMode!=eMode2 ? SQLITE_BUSY : rc);
dan7c246102010-04-12 19:00:29 +00002945}
2946
drh7ed91f22010-04-29 22:34:07 +00002947/* Return the value to pass to a sqlite3_wal_hook callback, the
2948** number of frames in the WAL at the point of the last commit since
2949** sqlite3WalCallback() was called. If no commits have occurred since
2950** the last call, then return 0.
2951*/
2952int sqlite3WalCallback(Wal *pWal){
dan8d22a172010-04-19 18:03:51 +00002953 u32 ret = 0;
drh7ed91f22010-04-29 22:34:07 +00002954 if( pWal ){
2955 ret = pWal->iCallback;
2956 pWal->iCallback = 0;
dan8d22a172010-04-19 18:03:51 +00002957 }
2958 return (int)ret;
2959}
dan55437592010-05-11 12:19:26 +00002960
2961/*
drh61e4ace2010-05-31 20:28:37 +00002962** This function is called to change the WAL subsystem into or out
2963** of locking_mode=EXCLUSIVE.
dan55437592010-05-11 12:19:26 +00002964**
drh61e4ace2010-05-31 20:28:37 +00002965** If op is zero, then attempt to change from locking_mode=EXCLUSIVE
2966** into locking_mode=NORMAL. This means that we must acquire a lock
2967** on the pWal->readLock byte. If the WAL is already in locking_mode=NORMAL
2968** or if the acquisition of the lock fails, then return 0. If the
2969** transition out of exclusive-mode is successful, return 1. This
2970** operation must occur while the pager is still holding the exclusive
2971** lock on the main database file.
dan55437592010-05-11 12:19:26 +00002972**
drh61e4ace2010-05-31 20:28:37 +00002973** If op is one, then change from locking_mode=NORMAL into
2974** locking_mode=EXCLUSIVE. This means that the pWal->readLock must
2975** be released. Return 1 if the transition is made and 0 if the
2976** WAL is already in exclusive-locking mode - meaning that this
2977** routine is a no-op. The pager must already hold the exclusive lock
2978** on the main database file before invoking this operation.
2979**
2980** If op is negative, then do a dry-run of the op==1 case but do
dan8c408002010-11-01 17:38:24 +00002981** not actually change anything. The pager uses this to see if it
drh61e4ace2010-05-31 20:28:37 +00002982** should acquire the database exclusive lock prior to invoking
2983** the op==1 case.
dan55437592010-05-11 12:19:26 +00002984*/
2985int sqlite3WalExclusiveMode(Wal *pWal, int op){
drh61e4ace2010-05-31 20:28:37 +00002986 int rc;
drhaab4c022010-06-02 14:45:51 +00002987 assert( pWal->writeLock==0 );
dan8c408002010-11-01 17:38:24 +00002988 assert( pWal->exclusiveMode!=WAL_HEAPMEMORY_MODE || op==-1 );
dan3cac5dc2010-06-04 18:37:59 +00002989
2990 /* pWal->readLock is usually set, but might be -1 if there was a
2991 ** prior error while attempting to acquire are read-lock. This cannot
2992 ** happen if the connection is actually in exclusive mode (as no xShmLock
2993 ** locks are taken in this case). Nor should the pager attempt to
2994 ** upgrade to exclusive-mode following such an error.
2995 */
drhaab4c022010-06-02 14:45:51 +00002996 assert( pWal->readLock>=0 || pWal->lockError );
dan3cac5dc2010-06-04 18:37:59 +00002997 assert( pWal->readLock>=0 || (op<=0 && pWal->exclusiveMode==0) );
2998
drh61e4ace2010-05-31 20:28:37 +00002999 if( op==0 ){
3000 if( pWal->exclusiveMode ){
3001 pWal->exclusiveMode = 0;
dan3cac5dc2010-06-04 18:37:59 +00003002 if( walLockShared(pWal, WAL_READ_LOCK(pWal->readLock))!=SQLITE_OK ){
drh61e4ace2010-05-31 20:28:37 +00003003 pWal->exclusiveMode = 1;
3004 }
3005 rc = pWal->exclusiveMode==0;
3006 }else{
drhaab4c022010-06-02 14:45:51 +00003007 /* Already in locking_mode=NORMAL */
drh61e4ace2010-05-31 20:28:37 +00003008 rc = 0;
3009 }
3010 }else if( op>0 ){
3011 assert( pWal->exclusiveMode==0 );
drhaab4c022010-06-02 14:45:51 +00003012 assert( pWal->readLock>=0 );
drh61e4ace2010-05-31 20:28:37 +00003013 walUnlockShared(pWal, WAL_READ_LOCK(pWal->readLock));
3014 pWal->exclusiveMode = 1;
3015 rc = 1;
3016 }else{
3017 rc = pWal->exclusiveMode==0;
dan55437592010-05-11 12:19:26 +00003018 }
drh61e4ace2010-05-31 20:28:37 +00003019 return rc;
dan55437592010-05-11 12:19:26 +00003020}
3021
dan8c408002010-11-01 17:38:24 +00003022/*
3023** Return true if the argument is non-NULL and the WAL module is using
3024** heap-memory for the wal-index. Otherwise, if the argument is NULL or the
3025** WAL module is using shared-memory, return false.
3026*/
3027int sqlite3WalHeapMemory(Wal *pWal){
3028 return (pWal && pWal->exclusiveMode==WAL_HEAPMEMORY_MODE );
3029}
3030
dan5cf53532010-05-01 16:40:20 +00003031#endif /* #ifndef SQLITE_OMIT_WAL */