2008-10-28 00:12 -!- FelipeS(~Felipe@r77h15.res.gatech.edu) has joined #tux3
2008-10-28 03:22 -!- RazvanM(~RazvanM@pool-151-196-126-202.balt.east.verizon.net) has joined #tux3
2008-10-28 04:27 -!- pgquiles(~pgquiles@19.Red-83-44-236.dynamicIP.rima-tde.net) has joined #tux3
2008-10-28 09:10 -!- mingming(~mingming@32.97.110.51) has joined #tux3
2008-10-28 09:28 -!- FelipeS(~Felipe@r77h15.res.gatech.edu) has joined #tux3
2008-10-28 09:47 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #tux3
2008-10-28 11:03 -!- flips(~phillips@phunq.net) has joined #tux3
2008-10-28 11:15 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #tux3
2008-10-28 12:02 -!- mingming_(~mingming@32.97.110.55) has joined #tux3
2008-10-28 12:48 -!- pgquiles(~pgquiles@19.Red-83-44-236.dynamicIP.rima-tde.net) has joined #tux3
2008-10-28 14:37 -!- mingming(~mingming@32.97.110.51) has joined #tux3
2008-10-28 14:39 -!- pgquiles_(~pgquiles@19.Red-83-44-236.dynamicIP.rima-tde.net) has joined #tux3
2008-10-28 15:18 -!- tim_dimm(~timothyhu@cpe-76-90-98-247.socal.res.rr.com) has joined #tux3
2008-10-28 15:39 -!- FelipeS(~Felipe@lawn-128-61-20-25.lawn.gatech.edu) has joined #tux3
2008-10-28 19:22 -!- RazvanM(~razvan@pool-151-196-126-202.balt.east.verizon.net) has joined #tux3
2008-10-28 19:23 <RazvanM> 
ACTION is macbookless this time :|

2008-10-28 19:39 <hirofumi> hi
2008-10-28 19:43 -!- RalucaM(~ral@pool-151-196-126-202.balt.east.verizon.net) has joined #tux3
2008-10-28 19:43 <RalucaM> hi
2008-10-28 19:43 <hirofumi> hi RalucaM 
2008-10-28 19:49 <bh> folks
2008-10-28 19:50 <hirofumi> hi
2008-10-28 19:56 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-10-28 20:08 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-10-28 20:09 <MaZe> miss anything?
2008-10-28 20:09 <MaZe> (my laptop just crashed on me twice in a row, I'm guessing because of a bad power supply...
2008-10-28 20:09 <RalucaM> nope
2008-10-28 20:09 <RalucaM> is today's class canceled?
2008-10-28 20:10 <MaZe> I would think not.... but maybe?
2008-10-28 20:10 <MaZe> flips: ping
2008-10-28 20:10 <RalucaM> :)
2008-10-28 20:11 <MaZe> although to be fair with the latest round of kernel+nvidia+madwifi upgrades the machine seems much less stable then it was in the past... so maybe it wasn't the power supply just bad luck.  Still twice in 10 minutes really takes the cake (the running tally is around thrice in the previous two weeks)
2008-10-28 20:12 <MaZe> and before that it was roughly once every 2-3 weeks
2008-10-28 20:12 <RalucaM> oh
2008-10-28 20:12 <RalucaM> what linux are you using?
2008-10-28 20:13 <RalucaM> debian was always quite stable for me :P
2008-10-28 20:13 <MaZe> fc9 with kernel from koji, newest nvidia, madwifi from svn head
2008-10-28 20:14 <RalucaM> I see
2008-10-28 20:14 <MaZe> it used to only lock up occasionally (rarely) during suspend to ram
2008-10-28 20:14 <MaZe> but now it's locked up a few times while I was typing on it... which is new
2008-10-28 20:14 <MaZe> maybe I'll leave it in memtest86 overnight
2008-10-28 20:14 <RalucaM> hm... maybe it's a hw issue...
2008-10-28 20:15 -!- FelipeS(~Felipe@r77h15.res.gatech.edu) has joined #tux3
2008-10-28 20:15 <MaZe> well, the wireless driver is binary hal + opensource code, and it _is_ buggy
2008-10-28 20:15 <MaZe> 2.6.27 supposedly fixes that
2008-10-28 20:15 <MaZe> with the ath9k driver
2008-10-28 20:15 <MaZe> but I'm still on 2.6.26.7
2008-10-28 20:16 <MaZe> I just wish I got kernel crashdumps or something out of this
2008-10-28 20:17 <MaZe> I should probably figure out how to use the kdump kernel stuff
2008-10-28 20:17 <RalucaM> :)
2008-10-28 20:18 <hirofumi> IIRC, /etc/sysconfig/kdump
2008-10-28 20:18 <MaZe> file not present
2008-10-28 20:18 <MaZe> probably have to install something first
2008-10-28 20:18 <MaZe> found an fc6 wiki
2008-10-28 20:35 <flips> blame timothy
2008-10-28 20:36 <hirofumi> hi
2008-10-28 20:36 <flips> ok, shall we start a little late?
2008-10-28 20:37 <hirofumi> folks?
2008-10-28 20:37 <MaZe> ah it is ;-)
2008-10-28 20:37 <RazvanM> ponk
2008-10-28 20:37 <MaZe> ok
2008-10-28 20:37 <RalucaM> works for me
2008-10-28 20:37 <RazvanM> later works for me too :D
2008-10-28 20:38 <MaZe> and here I was about ready to get synergy between laptop and home projector working
2008-10-28 20:38 <flips> so what would we rather look at: 1) the get_block question above 2) mysteries of rename?
2008-10-28 20:38 <MaZe> rename
2008-10-28 20:39 <flips> rename it is
2008-10-28 20:40 <bh> hey flips 
2008-10-28 20:40 <flips> let's go find where it happens
2008-10-28 20:40 <flips> fs/namei.c
2008-10-28 20:40 <bh> maybe tux3 rock the world ;)
2008-10-28 20:41 <hirofumi> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L2582
2008-10-28 20:41 <flips> beat me ;)
2008-10-28 20:41 <flips> by 2 seconds
2008-10-28 20:41 <hirofumi> or sys_rename
2008-10-28 20:41 <flips> most of this is permission checking and locking
2008-10-28 20:42 <flips> may_create, may_delete, just checking perms
2008-10-28 20:43 <flips> http://lxr.linux.no/linux+v2.6.26.6/fs/namei.c#L2671 vfs_rename_other - everything but directories
2008-10-28 20:43 <flips> i_mutex is the synchronizer
2008-10-28 20:44 <MaZe> what's last_type ?
2008-10-28 20:44 <flips> type of the last segment in the path, i.e., the file
2008-10-28 20:45 <flips> oops, I missed a huge item in vfs_rename
2008-10-28 20:45 <flips> lock_rename
2008-10-28 20:45 <MaZe> I'm still parsing sys_renameat ;-)
2008-10-28 20:46 <flips> oh you started at the syscall
2008-10-28 20:46 <flips> not much happening there
2008-10-28 20:46 <flips> I'm wondering what that mutex_lock_nested is
2008-10-28 20:46 <flips> new one for me, let's see how far back it goes
2008-10-28 20:47 <hirofumi> lockdep stuff
2008-10-28 20:47 <hirofumi> it's just mutex_lock without lockdep warning
2008-10-28 20:47 <flips> ah, so it's just a mutex
2008-10-28 20:47 <flips> ok
2008-10-28 20:47 <MaZe> the dentry gets created in it
2008-10-28 20:48 <flips> don't think so
2008-10-28 20:49 <flips> I should have started at do_rename actually
2008-10-28 20:49 <hirofumi> we are in do_rename now?
2008-10-28 20:49 <flips> the first big event is the path_lookup
2008-10-28 20:49 <hirofumi> ok
2008-10-28 20:49 <flips> yes, we went back
2008-10-28 20:49 <flips> to see where the dentries come from
2008-10-28 20:49 <MaZe> ugh, lock_rename
2008-10-28 20:49 <flips> I jumped to the run stuff first ;)
2008-10-28 20:50 <flips> ok, in linux whenever you have an open file you have a dentry for it
2008-10-28 20:50 <MaZe> right, but on rename the dest might not exist
2008-10-28 20:50 <MaZe> 'might' ;-)
2008-10-28 20:50 <flips> so the vfs can return the dentry as a handle for a directory/file
2008-10-28 20:50 <hirofumi> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L2625
2008-10-28 20:51 <flips> right, a dentry can exist without the underlying object existing
2008-10-28 20:51 <MaZe> #2679
2008-10-28 20:51 <flips> shall we go into path_lookup?
2008-10-28 20:51 <flips> buckle up your complexity belt
2008-10-28 20:51 <flips> http://lxr.linux.no/linux+v2.6.26.6/fs/namei.c#L1133
2008-10-28 20:52 <MaZe> you mean lookup_hash ? or something else
2008-10-28 20:52 <hirofumi> oh, 2.6.26
2008-10-28 20:52 <flips> ah, sorry
2008-10-28 20:52 <flips> I'll move to 2.6.27
2008-10-28 20:52 <flips> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L1045
2008-10-28 20:53 <flips> like all the path functions, it eventually calls path_walk
2008-10-28 20:53 <MaZe> this code
2008-10-28 20:53 <MaZe> is totally different in 2.6.27
2008-10-28 20:53 <MaZe> refactored at least
2008-10-28 20:53 <flips> really?
2008-10-28 20:54 <MaZe> sys_renameat is huge
2008-10-28 20:54 <MaZe> there's no do_rename
2008-10-28 20:55 <flips> true
2008-10-28 20:55 <hirofumi> it was killed by Al's cleanup
2008-10-28 20:55 <MaZe> yeah, it needed it
2008-10-28 20:56 <MaZe> ok, let's pick a version
2008-10-28 20:56 <flips> do_rename was close to renameat
2008-10-28 20:56 <MaZe> cause that's probably why this didn't make a lot of sense to me ;-)
2008-10-28 20:56 <flips> so it's been made renameat
2008-10-28 20:56 <flips> well we started a level below
2008-10-28 20:56 <flips> ok where were we
2008-10-28 20:57 <flips> ah, and there are new names for the walk functions
2008-10-28 20:57 <RazvanM> lookup_hash
2008-10-28 20:57 <flips> do_path_lookup
2008-10-28 20:57 <hirofumi> sys_renameat -> user_path_lookup -> do_path_lookup?
2008-10-28 20:57 <flips> where do you see lookup_hash?
2008-10-28 20:57 <RazvanM> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L2679
2008-10-28 20:58 <flips> hirofumi, yes
2008-10-28 20:58 -!- tim_dimm(~timothyhu@cpe-76-90-98-247.socal.res.rr.com) has joined #tux3
2008-10-28 20:58 <flips> ok, more refactoring
2008-10-28 20:59 <flips> this was previously done within the walk I suppose
2008-10-28 21:00 <flips> nameidata
2008-10-28 21:00 <flips> the struct that acts as scratchpad for this family of functions
2008-10-28 21:01 <flips> in some cases is abused as a substitute for function parameters
2008-10-28 21:01 <flips> anyway, the new factoring returns a nameidata instead of dentry
2008-10-28 21:01 <flips> and the lookup_hash converts the name in the nameidata to a dentry
2008-10-28 21:02 <flips> let's check out nameidata
2008-10-28 21:02 <MaZe> :-)
2008-10-28 21:03 <MaZe> this must be partially for inotify
2008-10-28 21:03 <flips> it was around long before inotify was
2008-10-28 21:03 <flips> http://lxr.linux.no/linux+v2.6.27/include/linux/namei.h#L18
2008-10-28 21:04 <flips> saved_names... never looked at that
2008-10-28 21:04 <MaZe> L27 though
2008-10-28 21:04 <MaZe> probably proc support
2008-10-28 21:04 <flips> probably is how symlink traversal was made nonrecursive
2008-10-28 21:05 <MaZe> so you can get meaningful dumps of fd's
2008-10-28 21:05 <flips> you get that by following parent links in dentries
2008-10-28 21:06 <flips> the "path" in the nameidata... a little oddly named
2008-10-28 21:06 <RazvanM> http://lxr.linux.no/linux+v2.6.27/include/linux/path.h#L7
2008-10-28 21:06 <flips> it's a dentry/mount pair
2008-10-28 21:07 <flips> not immediately obviously what the mount part is for
2008-10-28 21:07 <flips> anyway, I wanted to do rename this time
2008-10-28 21:07 <flips> not path walk, which I need to review first
2008-10-28 21:08 <flips> it keeps changing and it was complex to begin with
2008-10-28 21:08 <MaZe> (side note) an auto-parser which would figure out and auto-annotate code, with a comment, containing where it gets called from and what it calls, what locks are held before and after and during would be useful....
2008-10-28 21:08 <flips> have it ready by friday?
2008-10-28 21:08 <RazvanM> :D
2008-10-28 21:09 <MaZe> ;-)
2008-10-28 21:09 <flips> anyway, suffice to say for now that path_walk scans off each segment of the / separated path and looks for a dentry
2008-10-28 21:09 <flips> if it doesn't find one, it calls the filesystem
2008-10-28 21:10 <hirofumi> inode->i_op->lookup
2008-10-28 21:10 <flips> if the filesystem says its a symlink, which yields a new path
2008-10-28 21:10 <flips> the details a wickely complex
2008-10-28 21:10 <flips> for various reasons
2008-10-28 21:10 <flips> including nfs
2008-10-28 21:10 <flips> enough on path_walk for now
2008-10-28 21:10 <hirofumi> yes
2008-10-28 21:11 <flips> let's get back to the rename locking
2008-10-28 21:12 <flips> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L2657
2008-10-28 21:13 <flips> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L1462 <- lock)_rename
2008-10-28 21:13 <flips> pretty simple
2008-10-28 21:13 <flips> needs to lock in the order parent, child
2008-10-28 21:14 <flips> to avoid deadlock
2008-10-28 21:14 <MaZe> per fs sb lock 
2008-10-28 21:14 <flips> yes that too
2008-10-28 21:15 <flips> so we have the per-sb lock, and we will take a lock on each directly
2008-10-28 21:15 <flips> directory
2008-10-28 21:15 <flips> first, walk the chain of dentries up from one directory to the root, looking for the second directory
2008-10-28 21:15 <flips> if we find it, we know the second is ancestor of the first, so take the second lock first
2008-10-28 21:16 <flips> otherwise, do the same in the other direction
2008-10-28 21:16 <flips> if neither is ancestor of the other, the order doesn't matter
2008-10-28 21:16 <MaZe> why do we still have parent/child locks if there's no parent/child relation?
2008-10-28 21:16 <flips> though the _nested syntax doesn't make that obvious
2008-10-28 21:17 <flips> the answer to that will be found by looking at mutex_lock_nested
2008-10-28 21:18 <flips> the int subclass is only advisory
2008-10-28 21:18 <MaZe> oh the parameter is merely for run-time lock checking
2008-10-28 21:18 <flips> for automatically checking lock dependencies
2008-10-28 21:18 <flips> I suppose there should be a "no dependency" value, don't know why there isn't
2008-10-28 21:19 <MaZe> I think the PARENT CHILD are just arbitrary strings
2008-10-28 21:19 <MaZe> which usually happen to actually match what the locks labeled with them are used for
2008-10-28 21:19 <MaZe> NORMAL, PARENT, CHILD, XATTR, QUOTA
2008-10-28 21:20 <flips> #  define mutex_acquire(l, s, t, i)             lock_acquire(l, s, t, 0, 2, NULL, i) <- ooh, ugly
2008-10-28 21:21 <flips> I'm having a hard time believing that all that debugging stuff doesn't create runtime overhead when not used
2008-10-28 21:21 <MaZe> does mutex_lock sleep till it gets the lock?
2008-10-28 21:21 <flips> yes
2008-10-28 21:21 <MaZe> I'd guess it can get compiled out
2008-10-28 21:21 <MaZe> via the preprocessor or compiler opt
2008-10-28 21:22 <flips> I'll take that on faith
2008-10-28 21:22 <flips> I don't see the compiler removing all the extra parameters
2008-10-28 21:22 <flips> like the subclass
2008-10-28 21:22 <MaZe> anyway, I think I understand (un)lock_rename
2008-10-28 21:22 <flips> right
2008-10-28 21:22 <flips> we can look at lockdep another time
2008-10-28 21:23 <flips> useful facility that I have never used
2008-10-28 21:23 <flips> 
ACTION doesn't make locking errors

2008-10-28 21:23 <flips> usually
2008-10-28 21:24 <flips> ok, a couple more reality checks then into vfs_rename
2008-10-28 21:24 <MaZe> locking is just a matter of good design ;-)
2008-10-28 21:24 <MaZe> which is much easier when you're writing your own code from scratch
2008-10-28 21:25 <flips> I left off the smilely above
2008-10-28 21:25 <flips> everybody makes locking errors
2008-10-28 21:25 <MaZe> (since half the problem is knowing what the design is)
2008-10-28 21:26 <flips> vfs_rename is all just permission checks
2008-10-28 21:26 <flips> as we saw before
2008-10-28 21:26 <flips> the rename_other vs rename_directory
2008-10-28 21:26 <MaZe> why don't we abort here on trap != NULL ?
2008-10-28 21:26 <MaZe> back at 2657.
2008-10-28 21:26 <MaZe> oh nevermind
2008-10-28 21:27 <MaZe> we're still dealing with the dirs the stuff we're renaming are in, not the stuff we're renaming itself
2008-10-28 21:27 <flips> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L2567 <- we need to get yet another lock here
2008-10-28 21:27 <flips> the lock on a filesystem object we are about to implcitly unlink
2008-10-28 21:28 <MaZe> shouldn't this be a mutex_lock_nested call?
2008-10-28 21:28 <flips> doesn't need to be order with respect to source or dest dirs
2008-10-28 21:28 <flips> ordered
2008-10-28 21:29 <flips> on with respect to the sb rename mutex
2008-10-28 21:29 <flips> only
2008-10-28 21:30 <flips> anyway, we call the fs ->rename method and that's about all there is to that
2008-10-28 21:30 <flips> the fs can happiily work away knowing that all the needed locks are already taken
2008-10-28 21:31 <flips> oh can imagine bottlenecks here with mass renames
2008-10-28 21:31 <flips> but since one doesn't really see mass renames, it's not a big issue
2008-10-28 21:31 <flips> now rename_dir
2008-10-28 21:32 <hirofumi> oh, d_move?
2008-10-28 21:32 <MaZe> and renames within the same directory don't use the per-fs-sb lock
2008-10-28 21:32 <flips> ah right
2008-10-28 21:32 <flips> the dentry cache has to be updated to reflect what the filesystem did to the backing store
2008-10-28 21:33 <flips> good observation
2008-10-28 21:34 <flips> _other and _dir are almost identical
2008-10-28 21:34 <flips> probably should be one function
2008-10-28 21:36 <flips> the new_dentry is preemptively unhashed for some reason
2008-10-28 21:37 <flips> something of a mystery why
2008-10-28 21:37 <flips> this code really suffers from being nearly devoid of comments
2008-10-28 21:38 <MaZe> rehashed later on
2008-10-28 21:38 <flips> without explanation
2008-10-28 21:39 <flips> homework
2008-10-28 21:39 <hirofumi> maybe, comment of dentry_unhash
2008-10-28 21:39 <flips> homework: "wtf is the unhash/rehash in vfs_rename_dir all about?"
2008-10-28 21:39 <MaZe> I think this is how the target gets deleted?
2008-10-28 21:40 <hirofumi> http://lxr.linux.no/linux+v2.6.27/fs/namei.c#L2110
2008-10-28 21:40 <flips> see rename_other for a target also being unlinked
2008-10-28 21:40 <MaZe> after all a rename will move the source, but kill the destination
2008-10-28 21:40 <MaZe> hence S_DEAD
2008-10-28 21:42 <flips>  151#define S_DEAD          16      /* removed, but still open directory */
2008-10-28 21:42 <flips> http://lxr.linux.no/linux+v2.6.27/include/linux/fs.h#L151
2008-10-28 21:43 <flips> what that's special to directories is not clear
2008-10-28 21:43 <flips> regular files can also be removed but still open
2008-10-28 21:43 <hirofumi> if S_DEAD, we can't lookup anymore
2008-10-28 21:44 <MaZe> why is it even in the hash then?
2008-10-28 21:44 <flips> both good points
2008-10-28 21:45 <hirofumi> may be just for d_move?
2008-10-28 21:45 <MaZe> reiserfs refuses to read xattrs for a dead dir
2008-10-28 21:46 <MaZe> ENOENT automatically on readdir
2008-10-28 21:46 <flips> S_DEAD isn't used by very many filesystems
2008-10-28 21:46 <MaZe> (that last was vfs)
2008-10-28 21:46 <MaZe> see IS_DEADDIR
2008-10-28 21:47 <MaZe> may_create false in dead dir etc
2008-10-28 21:47 <hirofumi> yes
2008-10-28 21:48 <MaZe> can;t create child dentry for dead dir ( in lookup_hash)
2008-10-28 21:48 <flips> now, why is it still in the hash?
2008-10-28 21:49 <MaZe> laziness?
2008-10-28 21:50 <flips> somebody holds a count on it somehow?
2008-10-28 21:50 <flips> seems like a stretch
2008-10-28 21:50 <flips> well
2008-10-28 21:51 <flips> another day, another crufty bit of linux kernel
2008-10-28 21:52 <flips> rename is the ickiest of the vfs namespace functions
2008-10-28 21:52 <flips> the others will see clear by comparison
2008-10-28 21:53 <MaZe> hmm
2008-10-28 21:53 <MaZe> this didn't seem that bad
2008-10-28 21:53 <RazvanM> MaZe: could be worse, right? :D
2008-10-28 21:53 <MaZe> yup
2008-10-28 21:56 <RazvanM> 
ACTION says thanks for the lesson!

2008-10-28 21:56 <RazvanM> 
ACTION also goes to bed because he wake up very early today.

2008-10-28 21:56 <MaZe> yes, thanks!
2008-10-28 21:56 <MaZe> ok plug in power
2008-10-28 21:57 <hirofumi> dentry_unhash in rename seems to be just for strange fs
2008-10-28 21:58 <hirofumi>  if it cannot handle the case of removing a directory that is still in use by something else..
2008-10-28 21:59 <hirofumi> oh
2008-10-28 22:03 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-10-28 22:03 <MaZe> lol
2008-10-28 22:12 <flips> pushed the wrong button?
2008-10-28 22:13 <MaZe> nope
2008-10-28 22:14 <MaZe> ran out of battery power
2008-10-28 22:14 <flips> had to go find a socket and plug yourself in then
2008-10-28 22:14 <MaZe> nope
2008-10-28 22:14 <MaZe> instead of reaching for power cord
2008-10-28 22:14 <MaZe> I started typing
2008-10-28 22:14 <MaZe> and the system critical shut down
2008-10-28 22:15 <MaZe> since apparently the batter went from 30% to 2% in a couple seconds
2008-10-28 22:15 <MaZe> (clean shutdown though)
2008-10-28 23:14 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-10-28 23:47 <hirofumi> are you already starting to implement atomic commit?