This site is no longer maintained. Go to freebsd site.

Giant-less UFS with quotas for FreeBSD

As of 23-02-2006, FreeBSD marks ufs filesystems as not needing Giant, unless quotas support is compiled into the kernel. Proposed patch against 7-CURRENT aims to properly lock quota code and adds MNTK_MPSAFE flag to ufs mounts unconditionally.

Locking strategy

The following places were identified as needed to be properly locked:

• The dquot structures (see sys/ufs/ufs/quota.h) .
• dqhash, dqfreelist and struct dquot dq_cnt reference counter (see sys/ufs/ufs/ufs_quota.c).
• struct ufsmount' um_quotas, um_cred, um_btime, um_itime, um_qflags (see sys/ufs/ufs/ufsmount.h)
• the style of manipulation of struct dquot flags, especially DQ_LOCK, DQ_MOD and DQ_WANT
• assignment of the dquot structure pointer to i_dquot array of the inode

One global mutex (dqhlock) is used to protect dqhash, dqfreelist and dq_cnt, because they all manage busy/free/lookup operations for dquot'as.

Also, each dquot got individual mutex dq_lock to protect dq_flags. I.e., to safely access and manipulate dquot, the protocol shall be followed, where the operation starts by acquiring dq_lock mutex ownership. Then, thread shall wait in msleep(9) until DQ_LOCK flag is cleared by current owner. After that it is allowed to perform quick manipulations with dquot while dq_lock is held, or, if sleeping is possible, DQ_LOCK flag shall be set and mutex freed. For example,

	mtx_lock(&(dq->dq_lock));
	while (dq->dq_flags & DQ_LOCK) {
		dq->dq_flags |= DQ_WANT;
		(void) msleep(dq, &(dq->dq_lock), PINOD+1, "chkdq1", 0);
	}
	/* do something */
	dq->dq_flags |= DQ_MOD;
	mtx_unlock(&(dq->dq_lock));

To protect ufsmount data, um_lock mutex is used.

inode' i_dquot array is protected by lockmgr' vnode lock. As consequence, getinoquota(9) shall be called with exclusive lock on the vnode (except the case of NULLVP). The only caller that did not follow this (new) rule I found was ufs_access(), that could call getinoquota() with only shared lock held. I request upgrade of the vnode lock in ufs_access() before calling getinoquota() and downgrade lock after, if appropriate.

Patch

• 20060228 patch

  Initial public version.

• 20060302 patch

  Use vnode lock (instead of node interlock mutex) to protect dqget.

• 20060310 patch

  Patch refreshed against current. Fixed conflict.

• 20060329 patch

  Removed giant ownership assertions. Code is safe without giant (recent commits removed giant from all vfs syscalls).

• 20060330 patch

  Fixed two races in the dquot allocation code (dqget).

• 20060427 patch

  Merged with current changes for BO_NEEDSGIANT. Fixed deadlock when snapshot on the fs with active quotas is activated.

• 20060428 patch

  Fixed deadlock when mksnap_ffs and quotactl are called simultaneously. vn_start_write was called before quotactl, that seems to be not needed, and could lead to deadlock.

• 20060503 patch

  Fixes for parallel invocations of quotaon/quotaoff (establishing barrier by QTF_OPENING/QTF_CLOSING).

• 20060504 patch

  More fixes for parallel invocations of quotaon/quotaoff. Fix the userquota= syntax in fstab.

• 20060506 patch

  Merged with rev. 1.82 of ufs_quota.c, disabling quota accounting for snapshot files.

• 20060606 patch

  Fix the bug noted by Tor Egge. Aquire/release the Giant as needed when manipulating quotas file. Change was tested by placing quota file on the nfs-exported fs and artificially making the nfs client mp-unsafe.

$Id: index-quotagiant.html,v 1.21 2006/06/06 09:47:07 kostik Exp $