2008-09-08 00:13 hey 2008-09-08 00:13 how's it going ? 2008-09-08 00:14 ACTION just made it back to San Diego 2008-09-08 00:32 hey 2008-09-08 00:32 extended attributes are in process of being implemented 2008-09-08 00:32 only the disk format had any kind of design as of yesterday 2008-09-08 00:36 static struct xattrs foo = { .size = sizeof(foo), .list = { 2008-09-08 00:36 { .atom = 666, .size = 5, .data = "hello" }, 2008-09-08 00:36 { .atom = 777, .size = 6, .data = "world!" }, 2008-09-08 00:36 } }; 2008-09-08 02:46 the gates/seinfeld ad is awful, omgponies 2008-09-08 02:46 I couldn't keep watching it 2008-09-08 02:46 I've never seen anything that bad, not even close 2008-09-08 02:47 what were they thinking 2008-09-08 02:47 and how much did it cost ;-) 2008-09-08 02:48 -!- kbingham(~kbingham@193.132.141.186) has joined #tux3 2008-09-08 02:54 agreed 2008-09-08 03:21 ext3 delete speed really is pathetic 2008-09-08 03:22 that's one thing we must do much better 2008-09-08 03:22 and we will 2008-09-08 03:22 going to be fun when we get to the transaction handling part 2008-09-08 03:50 I found newer old skates! 2008-09-08 03:50 these are only a half size smaller than what I currently wear 2008-09-08 03:50 I think 2008-09-08 03:51 :-) 2008-09-08 03:51 it's spreading 2008-09-08 03:51 isn't it a little late over there? 2008-09-08 03:51 or early? 2008-09-08 03:51 same timezone as you silly 2008-09-08 03:51 oh, I got the idea it was mn 2008-09-08 03:52 ah, comcast's rdns is broken 2008-09-08 03:52 they kinda fit 2008-09-08 03:55 it's late all the same 2008-09-08 03:55 I'd better crash 2008-09-08 03:55 xattr hacking is going slowly 2008-09-08 03:56 slowly is better than not at all 2008-09-08 03:57 true 2008-09-08 03:57 it's going to be good I think 2008-09-08 03:58 I expect better that average attriburte cache performance 2008-09-08 04:00 with this much effort invested I don't doubt it 2008-09-08 04:00 ok these skates kinda hurt my bones after a little wearing 2008-09-08 04:00 but so do ski boots so what's new 2008-09-08 04:00 heh, it's tiny compared to the effort invested in any other fs I know of 2008-09-08 04:01 you can get decent skates online for $200 2008-09-08 04:01 right 2008-09-08 04:01 that don't hurt 2008-09-08 04:01 but I have no income at present 2008-09-08 04:01 student 2008-09-08 04:01 and these aren't too bad 2008-09-08 04:01 ah, go crazy on your skates then 2008-09-08 04:01 been them up 2008-09-08 04:01 I'm busily destroying mine 2008-09-08 04:01 need new ones pretty soon 2008-09-08 04:02 heh 2008-09-08 04:02 I'll take em for a spin tomorrow 2008-09-08 04:02 seattle's kind of hilly for skates 2008-09-08 04:02 I used to bike a bit, which seems easier on hills 2008-09-08 04:05 static struct xattrs foo = { .blob = { 2008-09-08 04:05 { .code = 666, .size = 6, .data = "hello" }, 2008-09-08 04:05 { .code = 777, .size = 7, .data = "world!" }, 2008-09-08 04:05 } }; 2008-09-08 04:06 cache form of immediate xattrs 2008-09-08 04:06 how do we know how big .blob is? 2008-09-08 04:06 struct xattrs { unsigned size; struct xattr { u16 code, size; char data[]; } blob[]; } PACKED; 2008-09-08 04:06 we count it when loading the inode 2008-09-08 04:06 right, your struct above didn't mention size 2008-09-08 04:06 there is a .size field 2008-09-08 04:06 right 2008-09-08 04:06 because C is too braindamaged to calculated it 2008-09-08 04:07 mhm 2008-09-08 04:07 does very much the wrong thing when you ask it to do something reasonable 2008-09-08 04:07 the linux kernel isn't written in ocaml though 2008-09-08 04:07 static struct xattrs foo = { .size = sizeof(foo), .blob = { <- ought to work 2008-09-08 04:07 but it does not 2008-09-08 04:07 jw, what does it do? 2008-09-08 04:08 it uses offsetof(struct xattrs, blob) 2008-09-08 04:08 some wanking about flexible arrays 2008-09-08 04:08 ah 2008-09-08 04:08 but the compiler bloody initialized the thing to a certain size and should use that as sizeof 2008-09-08 04:09 flaw in C imho 2008-09-08 04:09 C or GCC? 2008-09-08 04:09 committee misfeature 2008-09-08 04:09 standard 2008-09-08 04:12 anyway: 2008-09-08 04:12 xattr 666: 0x804a37c: 68 65 6c 6c 6f 00 "hello." 2008-09-08 04:12 xattr 777: 0x804a386: 77 6f 72 6c 64 21 00 "world!." 2008-09-08 04:12 dump_xattrs: zero length xattr 2008-09-08 04:12 that's what it does if there is garbage in it 2008-09-08 04:14 the plan is to walk across the inode attrs filling in a vector with pointer to each xattr encountered 2008-09-08 04:15 then add up the sizes of all the xattrs, allocate memory big enough, and copy the xattr data into the cache struct shown above 2008-09-08 04:17 the xattr cache vector will be a binary sized multiple 2008-09-08 04:17 so there is some slack space, some of the time, to store more xattrs in it 2008-09-08 04:18 but no big deal to just realloc to the next binary size up when necessary 2008-09-08 04:18 better than a linked list 2008-09-08 04:19 immediatel file data probably better go in this cache struct too 2008-09-08 04:19 though it can possibly also go in the page cache 2008-09-08 04:19 it will go there 2008-09-08 04:20 but if the page gets evicted, I think we want to be able to go back to the in-memory inode to repopulate the page cache, rather than going all the way back to the inode table block 2008-09-08 04:20 this means there will be some triple caching of immediate file data: 1) in the inode table block 2) in the xattr cache 3) in the page cache 2008-09-08 04:21 seems a little excessive 2008-09-08 04:21 got to think about that 2008-09-08 04:22 we also have double caching of xattr data: 1) in the inode table block and 2) in the page cache 2008-09-08 04:22 don't really like that either 2008-09-08 04:22 sorry 2008-09-08 04:23 we also have double caching of xattr data: 1) in the inode table block and 2) in the inode's xattr cache 2008-09-08 04:23 what might make more sense is to pin the inode table block in memory and have the inode point into the block buffer 2008-09-08 04:24 so if the inode is evicted by the vm, it drops its count on the inode table block, which may now be evicted if no other inode holds a count on it 2008-09-08 04:25 if we do that, xattr caching looks maybe more sensible 2008-09-08 04:26 if attributes are heavily versioned though, we may pin a log of inode table blocks in memory, which hold a very low density of data we are actually using 2008-09-08 04:26 --sensible 2008-09-08 04:27 I am probably overworried about this double/tripe caching issue 2008-09-08 04:27 we have that anyway with existing inode attributes 2008-09-08 04:29 the triple caching of immediate data can be improved to just double caching by not loading immediate file data into the attribute cache. I dunno. 2008-09-08 04:29 instead, we let a miss in the page cache to retrieve the inode table block and load the immediate data into the page cache 2008-09-08 04:30 ok, this is right 2008-09-08 04:31 then when we update the inode on disk, we store an immediate data attribute only if the page cache is a) dirty and b) still small enough to be an immediate data attribute 2008-09-08 07:17 -!- kbingham(~kbingham@193.132.141.186) has joined #tux3 2008-09-08 07:17 -!- pgquiles(~pgquiles@253.Red-83-44-239.dynamicIP.rima-tde.net) has joined #tux3 2008-09-08 07:17 -!- vcgomes(~vcgomes@li17-238.members.linode.com) has joined #tux3 2008-09-08 07:17 -!- data(~data@echo489.server4you.de) has joined #tux3 2008-09-08 07:17 -!- flips(~phillips@phunq.net) has joined #tux3 2008-09-08 07:17 -!- shapor(~shapor@yzf.shapor.com) has joined #tux3 2008-09-08 07:17 -!- konrad(~konrad@c-24-16-74-109.hsd1.mn.comcast.net) has joined #tux3 2008-09-08 07:17 -!- bh(~billh@ip68-107-26-122.sd.sd.cox.net) has joined #tux3 2008-09-08 09:00 -!- stargazr5(~gauravstt@59.95.36.185) has joined #tux3 2008-09-08 10:07 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #tux3 2008-09-08 11:24 so where were we 2008-09-08 11:24 xattrs 2008-09-08 11:25 so we are going to have this little xattr cache hanging off each inode, which is just a binary size chunk of memory a la kmalloc 2008-09-08 11:25 and xattrs get loaded into it when the inode is loaded, and written into it when somebody sets an xaddr, marking the inode 'xattr dirty' 2008-09-08 11:26 meaning the xattr cache has to be written back to the inode table block when the inode is saved/synced 2008-09-08 11:27 xattr atoms are stored on disk in directory format, specifically ext2 directory format for now. So we do an ext2_find_entry to resolve an xattr name to an atom on sys_getxattr or sys_setxattr 2008-09-08 11:28 then search the inode's xattr cache for that atom 2008-09-08 11:29 if ext2_find_entry fails for sys_setxattr, then we do ext2_create_entry 2008-09-08 11:30 for now, we are always going to load all xattrs when an inode is loaded and write all xattrs when the inode is saved 2008-09-08 11:30 later, especially with versioning, we don't want to load all xattrs every time 2008-09-08 11:31 so getxattr will first search the cache, then go to the inode table block if that fails 2008-09-08 11:31 and setxattr will have to scan the xattrs present in the inode to know which ones to keep and which ones to overwrite on save 2008-09-08 11:32 the new size of a saved inode will then not be completely determinable from examining the inode alone, as it is now 2008-09-08 11:32 so we get a little more complexity here, not too bad 2008-09-08 11:33 the initial implementation of load all, save all, no versioning will be pretty simple and fast 2008-09-08 11:35 always calling ext2_find_entry for each xaddr atom can be avoided by keeping a hash of xattr atoms, and we only do the find_entry on a miss in the xattr atom hash, or we always keep the xattr hash fully populated for now (there are usually only a few different kinds of xattrs) 2008-09-08 11:43 -!- elicriffield(~elicriffi@66.249.86.209) has joined #tux3 2008-09-08 11:58 hey 2008-09-08 12:07 hey 2008-09-08 12:09 hi eli 2008-09-08 12:09 it's xattrs day today ;-) 2008-09-08 12:09 oh fun :) 2008-09-08 12:09 yah, not the most exciting, exciting to some folks though 2008-09-08 12:10 man im just trying to stay awake today 2008-09-08 12:10 I guess a reasonable approach is emerging, should have something working by tomorrow say 2008-09-08 12:10 right, it's one of those for me too 2008-09-08 12:13 make iattr && ./iattr 2008-09-08 12:13 gcc -std=gnu99 -Wall -g iattr.c -o iattr 2008-09-08 12:13 xattr 666: 0x804a37c: 68 65 6c 6c 6f 00 "hello." 2008-09-08 12:13 xattr 777: 0x804a386: 77 6f 72 6c 64 21 00 "world!." 2008-09-08 12:13 two xattrs 2008-09-08 12:13 in an xcache ;-) 2008-09-08 12:13 now to wrap that with lots of xattr access yumminess 2008-09-08 12:20 -!- MaZe(~MaZe@216-239-45-4.google.com) has joined #tux3 2008-09-08 12:28 ACTION reads shapors diversity post 2008-09-08 13:04 -!- SEJeff(~jeff__@66.151.59.138) has joined #tux3 2008-09-08 13:42 -!- data(~data@echo489.server4you.de) has joined #tux3 2008-09-08 14:37 flips: 2008-09-08 14:37 You don't even need to play with autoconf, just do 2008-09-08 14:37 #define FUSE_USE_VERSION 26 2008-09-08 14:37 #include 2008-09-08 14:37 ... 2008-09-08 14:37 #if FUSE_VERSION >= 26 2008-09-08 14:38 fuse_main(argc, argv, &my_op, NULL); 2008-09-08 14:38 #else 2008-09-08 14:38 fuse_main(argc, argv, &my_op); 2008-09-08 14:38 #endif 2008-09-08 14:38 But all this is only important if you need some API features from 2008-09-08 14:38 2.6.x/2.7.x. Otherwise you can just use the old API unconditionally: 2008-09-08 14:38 #define FUSE_USE_VERSION 25 2008-09-08 14:38 #include 2008-09-08 15:00 -!- tim_dimm(~timothyhu@cpe-76-90-98-247.socal.res.rr.com) has joined #tux3 2008-09-08 16:56 ok, xattr cache lookup works 2008-09-08 16:56 konrad, we now do #define FUSE_USE_VERSION 27 2008-09-08 16:56 aj 2008-09-08 16:57 but good suggestion 2008-09-08 16:57 nice solution 2008-09-08 16:57 alright 2008-09-08 16:57 I should have xattr support in tux3.c by tomorrow 2008-09-08 16:58 then we can try it out in fuse 2008-09-08 16:58 fun 2008-09-08 16:58 I don't think we have functions for that yet 2008-09-08 16:58 not in tux3.c 2008-09-08 16:58 writing them now 2008-09-08 16:58 or rather, not in inode.c 2008-09-08 16:59 I meant in tux3fs.c 2008-09-08 16:59 maybe tux3fuse.c but I havn't looked at it much 2008-09-08 16:59 xcache_dump and xcache_lookup work, now writing xcache_update, which is considerably harder 2008-09-08 16:59 I wonder what passes for an xattr delete 2008-09-08 17:00 setxattr to empty? 2008-09-08 17:00 the low level fuse api has it 2008-09-08 17:14 alright 2008-09-08 17:15 touch: setting times of `tmp/abc': Function not implemented 2008-09-08 17:15 (high level api) 2008-09-08 17:17 I know 2008-09-08 17:17 don't know what it is 2008-09-08 17:17 sniffed at it a little 2008-09-08 17:17 libc braindamage it seems 2008-09-08 17:18 triggered by some combination of fuse things 2008-09-08 17:19 konrad, I suggest asking on the fuse list 2008-09-08 17:24 hm? I think we just havn't implemented one of the functions fuse wants us to 2008-09-08 17:24 might be 2008-09-08 17:25 Function 'name' not implemented would be much more informative 2008-09-08 17:31 hey tim_dimm 2008-09-08 17:31 hey flips 2008-09-08 17:35 shapor: ping 2008-09-08 17:40 xcache_delete works 2008-09-08 17:48 xcache_update works 2008-09-08 17:49 now some memory management to take care of changing the size of the xcache as necessary 2008-09-08 17:49 but first, a skate 2008-09-08 17:50 it's looking quite like the tux3 command will have get/set xattr by tomorrow 2008-09-08 19:43 -!- tim_dimm(~timothyhu@cpe-76-90-98-247.socal.res.rr.com) has joined #tux3 2008-09-08 19:59 -!- tim_dimm(~timothyhu@cpe-76-90-98-247.socal.res.rr.com) has joined #tux3 2008-09-08 22:53 time for a checkin 2008-09-08 23:16 been 30 hours :( 2008-09-08 23:33 folks