Reinitializer failing to run

[Will Noble]

Hi all,

I'm having a problem with reinitializers. The macro DEFINE-REINITIALIZER
defines a record variable, which might or might not be reachable from a
resumer function. When it is not, the reinitialization routine doesn't make
it into the saved heap image.

,open posix
(define (date)

(cond ((fork) => wait-for-child-process)
(else (exec "date"))))
; no values returned

,dump good.image

Writing good.image

,build (lambda (args) (date) 0) bad.image

Writing bad.image

Resuming good.image and calling DATE works as expected, but running the VM
with bad.image hangs it because the reinitializer from the posix-process
module that installs signal handlers doesn't run, causing the system to
ignore SIGCHLD signals.

Using ADD-INITIALIZTION-THUNK! instead of a reinitializer doesn't work
because when the thunk runs posix.so would not yet have been loaded back in.

It seems to me that we should be able to expect reinitializers to run. I'm
not sure what the best solution here is. We could use a separate
second-stage initialization thunk list to implement reinitializers, instead
of the current method (relying on record resumers). Or maybe there is a
natural way of ensuring that reinitializers are always reachable. If we find
an acceptable solution I'd be happy to code it up.

Best,
Will

[Taylor R Campbell]

Date: Mon, 29 Apr 2013 07:10:14 -0700
From: Will Noble <***@cow9.org>

It seems to me that we should be able to expect reinitializers to run. I'm
not sure what the best solution here is. We could use a separate
second-stage initialization thunk list to implement reinitializers, instead
of the current method (relying on record resumers). Or maybe there is a
natural way of ensuring that reinitializers are always reachable. If we find
an acceptable solution I'd be happy to code it up.

If you want to guarantee that the reinitializer has been included in
the image in some code you want to run, you can pass it to the NO-OP
procedure (from the BIG-UTIL structure, if I recall correctly) in that
code. E.g., you might do this at the start of WAIT-FOR-CHILD-PROCESS:

(define (wait-for-child-process pid)
(no-op sigchld-reinitializer)
...mess with pid...)

[Will Noble]

Post by Taylor R Campbell
If you want to guarantee that the reinitializer has been included in
the image in some code you want to run, you can pass it to the NO-OP
procedure (from the BIG-UTIL structure, if I recall correctly) in that
(define (wait-for-child-process pid)
(no-op sigchld-reinitializer)
...mess with pid...)

Thanks for the quick reply. No-op is a good trick to remember. I was just
thinking that if I did

(define (wait-for-child-process pid)
signals-reinitializer
...)

the variable reference might get removed.

However, in this case a ton of signals-related procedures in the POSIX
module would need this treatment. I think Mike would want to use some other
solution.

Best,
Will

[Taylor R Campbell]

Date: Tue, 30 Apr 2013 10:39:32 -0700
From: Will Noble <***@cow9.org>

However, in this case a ton of signals-related procedures in the POSIX
module would need this treatment. I think Mike would want to use some other
solution.

Would they really? I think only WAIT-FOR-CHILD-PROCESS and
FIND-NEXT-SIGNAL rely on it.

[Taylor R Campbell]

Date: Tue, 30 Apr 2013 18:30:51 +0000
From: Taylor R Campbell <***@mumble.net>

Date: Tue, 30 Apr 2013 10:39:32 -0700
From: Will Noble <***@cow9.org>

However, in this case a ton of signals-related procedures in the POSIX
module would need this treatment. I think Mike would want to use some other
solution.

Would they really? I think only WAIT-FOR-CHILD-PROCESS and
FIND-NEXT-SIGNAL rely on it.

FORK, too, I suppose, or anything that could create child processes,
so that we don't end up with zombies and nothing to reap them.

[Michael Sperber]

Post by Will Noble
Resuming good.image and calling DATE works as expected, but running the VM
with bad.image hangs it because the reinitializer from the posix-process
module that installs signal handlers doesn't run, causing the system to
ignore SIGCHLD signals.

Hmmm. It seems the problem is not so much the mechanism but that there
is no path from `date' to the reinitializer for the signal handlers.
Otherwise, it'd be likely that reinitializers would pollute images
dumped with a resumer procedure. Having said that, I'll have to think a
bit more about how to create such a path.

[Will Noble]

Post by Michael Sperber
Hmmm. It seems the problem is not so much the mechanism but that there
is no path from `date' to the reinitializer for the signal handlers.
Otherwise, it'd be likely that reinitializers would pollute images
dumped with a resumer procedure. Having said that, I'll have to think a
bit more about how to create such a path.

What I meant was that by changing the mechanism of reinitializers, we could
forget about the transitive closure of the resumer procedure and simply
include every reinitializer. I can think of a couple of reasons for doing
things this way.

For one thing, we already reload dynamically loaded libraries when resuming
from a resumer procedure-type dumped image. In this case, posix.so has
already polluted the image without the benefit of having signals work,
because of the missing reinitializer.

Second, the signal handler reinitializer doesn't do anything heavy. It only
arranges for the system to pick up on signals. To receive those signals the
user has to use the provided functions. Otherwise they would observe not
observe any effects.

Of course, it would be great if we don't get any unneeded reinitializers (or
code to reload unneeded share objects) in images. I think for that we would
need to have procedures refer to their containing packages, and have
packages refer to package-specific reinitializers.

Best,
Will

[Taylor R Campbell]

Date: Tue, 30 Apr 2013 10:08:02 -0700
From: Will Noble <***@cow9.org>

Of course, it would be great if we don't get any unneeded reinitializers (or
code to reload unneeded share objects) in images.

The dynamic externals reloading mechanism currently doesn't use
reinitializers because it has to run before lots of them. What if
each reinitializer had a list of those reinitializers that it depends
on, so that anything relying on dynamic externals would have to list
the dynamic externals renitializer? If nothing depends on dynamic
externals then the dynamic externals reinitializer could be GC'd and
wouldn't have to run.