COMPILATION

- On plow and squall, we've figured out how to link the agent .o files
  directly into the Java executable.  Make sure that we formalize this
  process so that we can do it whenever we need to.

- The configure script should not add -lstdc++, -lc, etc. to the link line
  if we are doing the linking through the compiler.

- On some platforms (e.g., the Serval cluster), the Java makefile is not
  creating the build/bin, build/bin/i686, etc., directories.

- Check the position of -lpthread on the link line

JDK

- The Linux and Solaris versions now have "connect" and "socket" stubs in 
  iomgr.c.  These same stubs should be added to the FreeBsd and Irix versions.

- We need to add an "accept" stub.

- patch.tcl should copy in the modified iomgr.c, io_md.c and iomgr.h.

SERVER 

- The single-threaded server should delete any pre-existing per-agent Java 
  class directory (upon agent arrival).

- The single-threaded server should delete the per-agent Java class directories
  on SIGTERM.

- The multi-threaded server uses a shared Java class directory, so it can't
  just delete the directory on agent arrival (although it can and should 
  delete it on SIGTERM).  Instead newpack.c must overwrite existing class
  files (as it is doing now), but it must use some kind of lock so that
  the files are not overwritten while some other agent is trying to read them.

- The server should be able to use a reserved TCP/IP port (if it is
  started as root).

EFFICIENCY

- Clean up the interaction among the five main classes -- Agent, AgentIO, 
  AgentMasks, AgentMeetings and AgentConnection -- and make this interaction
  much more efficient (i.e., less blocking, less brute-force search, etc.)

  = Add tclAgentIO.cc, javaAgentIO.cc, etc., which will implement
    language-specific IO classes that will be created through our existing
    factory mechanism

  = Move maskNotifierSetup, etc., into tclAgentIO.cc, since it is most closely
    associated with the IO loops

  = Move IncomingCallback into the IO files, since it relates exclusively to
    IO

  = Make the actual IO class aware of IncomingCallback so that it can tell
    IncomingCallback NOT to bother calling up to the language-specific
    layer if an IO operation is in progress (since if an IO operation is in
    progress, the IO operation will do the same checks that the callback
    would)

- Have only one SigioHandler instance that watches all the sockets (and hands
  ready sockets off to some small number of handler threads)

MASKS

? What is the right mask model?  Where should the incoming-message, etc.,
  queue be located?

RESTRICTIONS

- Strip out nested restrictions and for now, just have the machineGroup
  restriction

- Later add per-machine and self-imposed restrictions and low-resource
  events

MEETINGS

- Start meetings as nonblocking and switch to blocking exactly once.  This
  would require alternatives to tcpip_connect and tcpip_asyc_connect.  The
  alternative functions would not call tcpip_blocking and tcpip_nonblocking.
  (?)

MEETING SECURITY

- It must be impossible for an attacker to insert bogus data into the stream
  of messages/files sent across a meeting.  Solution: checksums

- We need to encrypt the messages/files sent across the meeting.  Solution:
  one-time-pad based on our cryptographically-secure random-number generator.

- The cryptographically-secure random-number generator must be turned into
  a class (rather than a package of functions with GLOBAL state), since each
  meeting needs its own generator.

- We need sequence numbers in the messages/files sent across a meeting so
  that we can detect replays (and rollback the CNRG to a known state).

- We might need meeting control messages so that one agent can ask the other
  agent to do a rollback, etc.

? Which of these things become unnecessary if we have SECURE TCP/IP in a lower
  layer?

SECURITY

- Single policy database that encompasses both use-defined and system resources.
  (?)

SCHEDULING

? What if too many agents arrive or wake up at once?

COMMUNICATION

- multicast ability (multicast groups inside the agent servers?) that authorized
  agents can use to wake up a set of waiting agents

- look into T/TCP

- We should set up the protocol so that only ONE side of the connection
  does the active close, thus limiting the number of TCP connections stuck
  in the TIME_WAIT state (right now both sides can end up doing an active
  close).  Before doing the work though, we should try to determine how
  many connections actually have both sides end up doing an active close,
  and then forget about it if it's only a small percentage.

COMMUNICATION CORRECTNESS

- tcpip_connect, tcpip_async_connect, unix_connect, unix_async_connect,
  tcpip_accept, and unix_accept could leave the socket in the wrong state
  (due to their calls to tcpip_blocking and tcpip_nonblocking).

- Most of the nonblocking functions in messaging/mesgTcpip.cc will actually
  block under Java since the stubs in iomgr.c all block.

COMMUNICATION SYMMETRY

- Use the Connection classes in the server

COMMUNICATION EFFICIENCY

- It would make sense to move some of the TCP/IP routines into the platform-
  specific layer.  For example, the Java routines would never need to switch
  to and from non-blocking mode, since the JVM puts all fds into nonblocking
  mode (and keeps them that way).

- It is a waste of time to call tcpip_asynchronous on the meeting sockets.
  Could this call be moved down into the single-threaded SigioHandler?

USELESS WORK

- We do not need to initialize TCL_LIBRARY in initcmd, etc., in tclBasicCmd.cc.

CODE ORGANIZATION

- Organize the tests subdirectory

- Merge genTransmit.h and genTransmit.cc into the new ConnectionDriver class

- Move the server message codes out of genMessage.h into the ServerInterface
  class

- Move the other message codes out of genMessage.h

- Finish distributing the genUtility.h functions into the appropriate modules

- We do not need FileConnection or FileCallbacks -- we can just implment
  class TclFileConnection: public Connection and pass instances of this new
  class to the meeting methods.

- Adjust the DynamicString and DynamicBuffer constructors (so that they are 
  implemented the same way)

- Adjust the copyOnWrite method in DynamicBuffer to match the copyOnWrite
  method in DynamicString

- Move servPointerList.cc, genDaemon.cc and genPipe.cc into server-support

- Java HandleMap could be simplified (if we decide to allow only one Agent
  instance per thread total)

PYTHON

- Make Python work again

- Implement submit and jump for Python

MEMORY USAGE

- each ServerAgent instance takes up too much space

THREADING

. Reference counting with a lock (as opposed to re-allocating memory) is
  worth it in a threaded environment.  It is about 4 times faster.  See the
  red notebook for details.

- With the new Slackware, the threaded server drops into some kind of
  infinite loop if we try to run it as a daemon.  Is the LinuxThreads
  controller thread getting blown away when we switch to being a daemon?

- We probably want a platMemoryImp.cc for pthreads-unix as well.

- Use the ReferenceCount class down in the platform hierarchy 
  (e.g., MonitorLock)

- Use FastMutexLock down in the platform hierarchy

- steps to thread-safety

  (1) DynamicString must be thread-safe (use FastMutexLock)
  (2) DynamicBuffer must be thread-safe (use FastMutexLock)
  (3) Hash must be thread-safe (use FastMutexLock)
  (4) BinaryData must be thread-safe (use FastMutexLock)
  (5) FileTable must be thread-safe
  (6) per-thread masks (only if an agent can have more than one thread)
  (7) per-thread restrictions (only if an agent can have more than one thread)
  (8) per-agent restrictions (in execution environments where we have more
      than one agent in the same process)

  For (1) through (4), remember that we only need to protect the reference
  counting (and copy-on-write) -- multiple readers/single writers is enforced
  at a higher level.

AGENT JAVA 

- We need to a better way to include/exclude classes in/from the state
  image (e.g., an additional argument to submit, jump and fork that
  specifies classes to exclude or include).  In addition, we might need
  to have four levels of inclusion: include nothing, include the values
  static variables, include the class file, and include both the static
  variables and the class file.

BUGS

- For Agent Java, the serialization code (in newpack.c) is not correctly
  serializing doubles or floats that are equal to -Infinity, +Infinity or
  NaN.

- For Agent Java, the serailization code (java_fscanf in newpack.c) can
  scan right off the end of the buffer if the buffer is constructed in the
  right way.

- Under Linux at least, we are not getting any core files when a server
  process aborts.

- Under Linux at least, no SIGIO is generated when the pipe connecting
  the agent to the server is closed from the server end - this means that
  we will not correctly detect a lost background connection unless we start
  doing some kind of complex polling.

- If a jumping Tcl agent dies during initialization, the Tcl interpreter goes
  into an infinite loop (because the slave interpreter is not empty, which
  prevents it from being deleted, etc.)

- If we kill a Java interpreter, we somehow end up calling MonitorLock::acquire
  from a thread that has no associated execution environment (thus EE() 
  returns NULL and the assertion in MonitorLock::acquire fails)

- If timeout is "large" in Agent Java, it sends up being treated as a timeout
  of 0.

- In the multi-threaded server, when a global wall-time violation occurs
  in an agent (and the associated interpreter exits), the server gets am
  "assert (Connection::status != Connected) failed" and then drops into an
  infinite loop when it tries to abort.

FAULT TOLERANCE

- We need an automatic agent-restart facility.

AGENT MODEL

? Do we need to allow agent_begin to register the agent with a remote machine,
  e.g., an agent starts on a Web server (via a CGI script), registers with
  an actual agent machine, etc.?

ADMINISTRATION

- A web interface to the server (e.g., a URL that lists all running agents,
  etc.) might be useful.
