17:29 [Users #openbsd-daily]
17:29 [@mulander] [ bsdtux] [ iomotoko] 
17:29 -!- Irssi: #openbsd-daily: Total of 3 nicks [1 ops, 0 halfops, 0 voices, 2 normal]
17:29 <@mulander> ------- code read: httpd --------
17:29 <@mulander> ** goal: find out how htpasswd is handled **
17:30 <@mulander> yesteday I read htpasswd.c which generates password files that web servers use to restrict access
17:30 <@mulander> httpd in base system handles this and I want to know how and what API it uses for handling the encrypted code (the old bcrypt or the new necrypt API)
17:31 <@mulander> the target is scoped as httpd itself i 7k lines of C, 2k lines of yacc and 0.8k lines of headers (~10k lines of code)
17:31 <@mulander> starting off by looking at the manpage
17:32 <@mulander> doing a search for htpasswd lands us immediately on the option
17:32 <@mulander> ---
17:32 -!- Irssi: Pasting 6 lines to #openbsd-daily. Press Ctrl-K if you wish to do this or Ctrl-C to cancel.
17:32 <@mulander>      [no] authenticate [realm] with htpasswd
17:32 <@mulander>              Authenticate a remote user for realm by checking the credentials
17:32 <@mulander>              against the user authentication file htpasswd.  The file name is
17:32 <@mulander>              relative to the chroot and must be readable by the www user.  Use
17:32 <@mulander>              the no authenticate directive to disable authentication in a
17:32 <@mulander>              location.
17:32 <@mulander> ---
17:33 <@mulander> ok so we confirmed it's handled
17:33 <@mulander> let's do a quick grep to see occurences of htpasswd in code
17:33 <@mulander> https://gist.github.com/mulander/05fd26d789ea778d9b1c0c451bdeec01
17:33 <@mulander> we see the manpage entry, first, we can skip that
17:34 <@mulander> then there's a parse.y file
17:34 <@mulander> which is probably an intermediate parsing declaration file
17:34 <@mulander> taking a look at Makefile to see how it's built
17:35 <@mulander> just defined as a target, wonder if the toolchain just handles that
17:36 <@mulander> right it's just a C file
17:37 <@mulander> it just handles reading the filename from the authenticate configuration
17:37 <@mulander> first interesting bits will be config.c which I assume grabs the data from the parsed file into a structure that can be reused in other parts of the code
17:37 <@mulander> opened up config.c for reading
17:37 <@mulander> 750 lines of C
17:37 <@mulander> we are not reading the whole of it
17:38 <@mulander> 3 occurences of htpasswd
17:38 <@mulander> first one inside
17:38 <@mulander> int config_setauth(struct httpd *env, struct auth *auth)
17:40 <@mulander> we have a httpd struct called nev and an auth struct
17:40 <@mulander> some code related to privsep iterating over PROC_MAC
17:40 <@mulander> my assumption is
17:41 <@mulander> the process needing the authentication setup is not the same process as the one parsing the code
17:41 <@mulander> hence this code iterates over the maximum amount of processes that could have been spawned
17:43 <@mulander> it skips the process if it already has an authentication defined or it is the currently running process
17:43 <@mulander> if not, it composes a message to send to the other process using inter process communication
17:44 <@mulander> quick check to see what proc compose is
17:44 -!- Irssi: Pasting 6 lines to #openbsd-daily. Press Ctrl-K if you wish to do this or Ctrl-C to cancel.
17:44 <@mulander> [mulander@napalm httpd]$ ag ^proc_compose
17:44 <@mulander> proc.c
17:44 <@mulander> 752:proc_compose_imsg(struct privsep *ps, enum privsep_procid id, int n,
17:44 <@mulander> 768:proc_compose(struct privsep *ps, enum privsep_procid id,
17:44 <@mulander> 775:proc_composev_imsg(struct privsep *ps, enum privsep_procid id, int n,
17:44 <@mulander> 790:proc_composev(struct privsep *ps, enum privsep_procid id,
17:44 <@mulander> defined in proc
17:44 <@mulander> calls proc_compose_imsg defined in same file
17:44 <@mulander> which iterates over a range of processes
17:44 <@mulander> calling imsg_compose_event
17:45 <@mulander> also defined in this file
17:45 <@mulander> which calls imsg_compose and then imsg_event_add
17:46 <@mulander> imsg_compose is part of imsg.h a system lib
17:46 <@mulander> man imsg_compose
17:46 <@mulander>      imsg_compose() is a routine which is used to quickly create and queue an
17:46 <@mulander>      imsg.  It takes the same parameters as the imsg_create(), imsg_add() and
17:46 <@mulander>      imsg_close() routines, except that only one ancillary data buffer can be
17:46 <@mulander>      provided.  This routine returns 1 if it succeeds, -1 otherwise.
17:46 <@mulander> imsg_event_add is slightly above
17:47 <@mulander> and it boils down to calling:
17:47 <@mulander> imsg_flush, event_del, event_set, event_add
17:47 <@mulander> man imsg_flush
17:47 <@mulander>      imsg_flush() is a function which calls msgbuf_write() in a loop until all
17:47 <@mulander>      imsgs in the output buffer are sent.  It returns 0 if it succeeds, -1
17:47 <@mulander>      otherwise.
17:48 <@mulander> event_del, event_set, event_add are all part of event.h so are system documented
17:48 <@mulander>      The event API provides a mechanism to execute a function when a specific
17:48 <@mulander>      event on a file descriptor occurs or after a given time has passed.
17:48 <@mulander> from man event_del ^
17:49 <@mulander> so the auth structure is indeed passed to a process using inter process communication
17:49 <@mulander> the other 2 occurences are in config_getauth
17:50 <@mulander> this receives the imsg sent from config_setauth
17:50 <@mulander> the auth tructure is compied out using memcpy from the passed in imsg
17:51 <@mulander> then auth_add is calaled on it
17:51 <@mulander> *called
17:51 <@mulander> we find wher auth_add is defined
17:51 <@mulander> [mulander@napalm httpd]$ ag ^auth_add
17:51 <@mulander> httpd.c
17:51 <@mulander> 1279:auth_add(struct serverauth *serverauth, struct auth *auth)
17:52 <@mulander> starting to read auth_add
17:53 <@mulander> we see calls to TAILQ functions, they are defined in sys/queue.h
17:53 <@mulander> man TAILQ_FOREACH
17:53 <@mulander>      These macros define and operate on four types of data structures: singly-
17:53 <@mulander>      linked lists, simple queues, lists, and tail queues.  All four structures
17:53 <@mulander>      support the following functionality:
17:53 <@mulander> in our case this is a tail queue.
17:54 <@mulander> since we can define more then one site, using more then one file then we have a queue of them
17:54 <@mulander> stored as serverauth
17:54 <@mulander> this just adds each auth to that queue
17:55 <@mulander> let's list all auth_functons
17:55 <@mulander> ag ^auth
17:55 <@mulander> httpd.c
17:55 <@mulander> 1279:auth_add(struct serverauth *serverauth, struct auth *auth)
17:55 <@mulander> 1299:auth_byid(struct serverauth *serverauth, uint32_t id)
17:55 <@mulander> 1312:auth_free(struct serverauth *serverauth, struct auth *auth)
17:56 <@mulander> auth_add, adds an entry, auth_free removes it from the queue and auth_byid just returns the auth identified by some id
17:56 <@mulander> lets lok where auth_byid is called as that's should be near where the file is used.
17:58 <@mulander> the single use seems to be in config.c
17:58 <@mulander> it's called in:
17:58 <@mulander> int config_getserver_auth(struct httpd *env, struct server_config *srv_conf)
17:59 <@mulander> it's fairly straightforward.
17:59 <@mulander> gets an httpd env, a server config and gets the authentication based on the id from the server config
17:59 <@mulander> let's see where that is called
18:00 <@mulander> also in config.c
18:00 <@mulander> first occurrence is just a forward declaration
18:00 <@mulander> second one is the function definition itself
18:00 <@mulander> third call occurs in config_getserver_config
18:01 <@mulander> and last one in config_getserver
18:01 <@mulander> which is a call handling IMSGs
18:02 <@mulander> going back to config_getserver_auth we see it's actually saved inside srv_conf->auth
18:02 <@mulander> srv_conf stands for server_config
18:02 <@mulander> so let's look when that value is read by searching ag srvv_conf->auth
18:03 <@mulander> https://gist.github.com/mulander/cefd76ce1b9e7a6b1762f0eb7b84e6d3
18:04 <@mulander> we will ignore matches in config
18:04 <@mulander> as we know they just set/pass values around
18:04 <@mulander> first server.c setting to NULL is just some initialization
18:04 <@mulander> no usage nearby
18:04 <@mulander> we are left with server_http.c where it seems the main usage happens.
18:04 <@mulander> let's go there
18:05 <@mulander> we find only a single match in:
18:05 <@mulander> int
18:05 <@mulander> server_http_authenticate(struct server_config *srv_conf, struct client *clt)
18:06 <@mulander> let's look at the code as it seems it has a bit more meat
18:06 <@mulander> const struct auth*auth = srv_conf->auth;
18:06 <@mulander> I keep 'auth' highlighted
18:06 <@mulander> to visually better see when that's used
18:07 <@mulander> there is some initial header checks first
18:07 <@mulander> looks like the header and pass are passed in via headers with basic auth
18:07 <@mulander> the first code splits them into ctl_user and ctl_pass
18:08 <@mulander> basic auth passes a 64 base encoded username:password
18:09 <@mulander> so the first strncmp finds the Basic header part
18:09 <@mulander> b64_pton decodes it from base64
18:09 <@mulander> into the decoded variable
18:10 <@mulander> it's then split into ctrl_pass bysetting it to the first character after :
18:10 <@mulander> now we get to our file
18:11 <@mulander> auth->auth_htpasswd stores the path to the file we generated with htpasswd
18:11 <@mulander> it's open read only
18:11 <@mulander> by now I highlight fp
18:11 <@mulander> file is read line by line split as user/pass
18:12 <@mulander> if there is no paswword, we skip the entry and explicit_bzero the read line (just in case)
18:12 <@mulander> if the username doesn't match, we skip the entry and also explicit_bzero
18:12 <@mulander> now we use crypt_checkpass
18:12 <@mulander> and that's the bit I wanted to know!
18:13 <@mulander> we know that htpasswd uses bcrypt api
18:13 <@mulander> as we read
18:13 <@mulander>      These functions are deprecated in favor of crypt_checkpass(3) and
18:13 <@mulander>      crypt_newhash(3).
18:15 <@mulander> now this means it might be sensible to replace the bcrypt API usage in htpasswd with the new crpt_* API
18:15 <@mulander> back to the code, we can see that when crypt_checkpass says the passwords match ret is set to 0
18:16 <@mulander> so let's do a quick look where server_http_authenticate is called.
18:16 <@mulander> also in server_http.c
18:16 <@mulander> one call in server_response
18:16 <@mulander> 	} else if (srv_conf->flags & SRVFLAG_AUTH &&
18:16 <@mulander> 	    server_http_authenticate(srv_conf, clt) == -1) {
18:16 <@mulander> 		server_abort_http(clt, 401, srv_conf->auth_realm);
18:16 <@mulander> 		return (-1);
18:17 <@mulander> the file has mixed indentination, that's worth remembering as it should be unified if we have something to  change here
18:17 <@mulander> the usage code is easy though
18:17 <@mulander> if tis entry has a server auth required
18:17 <@mulander> check the auth
18:18 <@mulander> if the auth failed, abort and send a 401 to the user
18:18 <@mulander> and that's it as the other occurence is just the function declaration and definition
18:18 <@mulander> ---------- DONE ----------