Almost ready for distribution...

NEW

@@ -1,29 +1,86 @@
-Major C code rewrite. Code is modular and extensible. Hopefully, next
-versions will include code for local domain sockets, file descriptors,
-pipes (on unix) and named pipes (on windows) as a bonus.
+What's New
 
-All functions provided by the library are in the namespace "socket".
-Functions such as send/receive/timeout/close etc do not exist anymore as
-stand alone functions. They are now only available as methods of the
-appropriate objects.
+Everything is new! Many changes for  2.0 happened in the C layer, 
+which has been almost completely rewritten. The code has been ported to 
+Lua 5.0 and greatly improved.  There have also been some API changes
+that made  the interface simpler and  more consistent. Here are  some of
+the changes that made it into version 2.0: 
 
-TCP has been changed to become more uniform.  First create an object, then 
-connect or bind if needed. Then use IO functions. The "socket.connect" and
-"socket.bind" functions are provided for simplicity, but they just call
-"socket.tcp" followed by the ":connect" or ":bind" methods.
+<> Major C code rewrite. Code is modular and extensible. Hopefully, other
+   developers will be motivated to provide code for SSL, local domain 
+   sockets, file descriptors, pipes (on Unix) and named pipes etc;
 
-All functions return a non-nil value as first return value if successful.
-All functions return whatever could be retrieved followed by error message 
-in case of error. The best way to check for errors is to check for the
-presence of an error message.  WARNING: The send function was affected.
+<> Everything that is exported by the library is exported inside 
+   namespaces. These should be obtained with calls to the
+   'require' function; 
 
-Better error messages and parameter checking.
+<> Functions such as
+   send/receive/timeout/close etc do not exist anymore as stand-alone
+   functions. They are now only available as methods of the appropriate
+   objects;
 
-UDP connected udp sockets can break association with peer by calling
-setpeername with address "*". 
+<> All functions return a non-nil value as first return value if successful.
+   All functions return 'nil' followed by error message
+   in case of error. This made the library much easier to use; 
 
-socket.sleep and socket.time are now part of the library and are 
-supported.
+<> Greatly reduced the number of times the C select is called 
+   during data transfers, by calling only on failure. This might 
+   improve a lot the maximum throughput;
+
+<> TCP has been changed to become more uniform.  It's possible to first
+   create a TCP object,
+   then connect or bind if needed, and finally use I/O functions. 
+   'socket.connect' and 'socket.bind' functions are still
+   provided for simplicity; 
+
+<> This allows for setting a timeout value before connecting;
+
+<> And also allows binding to a local address before connecting; 
+
+<> New 'socket.dns.gethostname' function and 'shutdown'
+   method;
+
+<> Better error messages and parameter checking;
+
+<> Should be interrupt safe; 
+
+<> UDP connected sockets can break association with peer by calling
+   'setpeername' with address ''*'';
+
+<> Sets returned by 'socket.select' are associative;
+
+<> Select checks if sockets have buffered data and returns immediately; 
+
+<> 'socket.sleep' and 'socket.time' are now part of the
+   library and are supported. They used to be available only when
+   LUASOCKET_DEBUG was defined, but it turns out they might be useful for
+   applications;
+
+<> 'socket.try' and 'socket.protect' provide a simple
+   interface to exceptions that proved very in the implementation of
+   high-level modules; 
+
+<> Socket options interface has been improved. TCP objects also 
+   support socket options and many new options were added.
+
+
+Lots of changes in the Lua modules, too! 
+
+<> Every module loads only the modules that it needs. There is no waste
+   of memory. LuaSocket core takes only 20k of memory;
+
+<> New MIME and LTN12 modules make all other modules much more powerful; 
+
+<> Support for multipart messages in the SMTP module;
+
+<> The old callback mechanism of FTP and HTTP has been replaced with LTN12 
+   sources and sinks, with advantage; 
+
+<> Common implementation for low-level FTP and SMTP;
+
+<> FTP, HTTP, and SMTP are implemented in multiple levels in such a way 
+   that users will have no problems extending the functionality to satisfy
+   personal needs; 
+
+<> SMTP knows how to perform LOGIN and PLAIN authentication. 
 
-Socket options interface has been improved and TCP now also supports socket
-options.

README

@@ -1,9 +1,11 @@
-This release is work in progress. It has been tested on WinXP, Mac OS X,
-SunOS and Linux. 
+This release is a "beta" version. It has been tested on WinXP, Mac OS X,
+SunOS and Linux. Although no major API changes should happen before the
+final version is released, please look for and report any bugs (or
+"features") you encounter. 
 
-In this version, all Lua code should be built into the binary. For that, you
-will need a working versions of luac and bin2c, both available with your
-Lua distribution. Check the makefile for details.
+For this version, all modules should be loaded with the provided "require"
+function, and the binaries should be compliled as shared libraries. Check
+the makefiles in the distribution and the readme in the 'etc' directory.
 
 Have fun,
 Diego Nehab.

etc/README

@@ -4,34 +4,42 @@ This directory contains code that is more useful than the examples. This code
     lua.lua
 
 These are modules to suport dynamic loading of LuaSocket by the stand alone
-Lua Interpreter with the use of the "require" function. For my Mac OS X
-system, I place all files in /Users/diego/tec/luasocket
-and set the following environment variables:
+Lua Interpreter with the use of new "require" and "requirelib" functions.
+For my Mac OS X box, for instance, I place all files in
+/Users/diego/tec/luasocket and set the following environment variables:
 
-    LUA_PATH=/Users/diego/tec/luasocket/?.lua
     LUA_INIT=@/Users/diego/tec/luasocket/lua.lua
-    LUA_FUNCNAME=?
-    LUA_LIBNAME=/Users/diego/tec/luasocket/?.dylib
+    LUA_PATH=/Users/diego/tec/luasocket/?.lua;?.lua
+    LUA_PATHLIB=/Users/diego/tec/luasocket/?.dylib;?.dylib
 
 With that, I can run any luasocket application with the command line:
 
-    lua -l socket <script>
+    lua <script>
 
+as long as the script uses "require" to load the needed namespaces.
 Much nicer than having to build a new executable just to initialize
 LuaSocket!
 
-
     tftp.lua                -- Trivial FTP client
 
 This module implements file retrieval by the TFTP protocol. Its main use
-is to test the UDP code, but someone might find it usefull.
+was to test the UDP code, but since someone found it usefull, I turned it
+into a module that is almost official (no uploads, yet).
+
+    dict.lua                -- Dict client
+
+The dict.lua  module started with  a cool  simple client  for the  DICT
+protocol, written by Luiz Henrique Figueiredo. This new version has been
+converted into a library, similar to the HTTP  and FTP libraries, that can
+be used from within  any luasocket application. Take  a look on the  source
+code and you will be able to figure out how to use it.
 
     get.lua                 -- file retriever
 
 This little  program is  a client  that uses  the FTP  and HTTP  code to
 implement a  command line  file graber. Just  run 
 
-	lua -l socket get.lua <remote-file> [<local-file>]
+	lua get.lua <remote-file> [<local-file>]
 
 to download  a remote file (either  ftp:// or http://) to  the specified
 local file.  The program  also prints  the download  throughput, elapsed
@@ -44,7 +52,7 @@ similar to check-links.pl by Jamie Zawinski, but uses all facilities of
 the LuaSocket library and the Lua language. It has not been thoroughly
 tested, but it should work. Just run
 
-	lua -l socket check-links.lua {<url>} > output
+	lua check-links.lua {<url>} > output
 
 and open the result to see a list of broken links. 
 

etc/dict.lua

@@ -4,78 +4,144 @@
 -- Author: Diego Nehab
 -- RCS ID: $Id$
 -----------------------------------------------------------------------------
+
+-----------------------------------------------------------------------------
+-- Load required modules
+-----------------------------------------------------------------------------
 local socket = require("socket")
+local url = require("url")
+local tp = require("tp")
 
-function get_status(sock, valid)
-	local line, err = sock:receive()
-	local code, par
-	if not line then sock:close() return err end
-	code = socket.skip(2, string.find(line, "^(%d%d%d)"))
-	code = tonumber(code)
-	if code ~= valid then return code end
-	if code == 150 then
-		par = tonumber(socket.skip(2, string.find(line, "^%d%d%d (%d*)")))
+-----------------------------------------------------------------------------
+-- Globals 
+-----------------------------------------------------------------------------
+HOST = "dict.org"
+PORT = 2628
+TIMEOUT = 10
+
+-----------------------------------------------------------------------------
+-- Low-level dict API
+-----------------------------------------------------------------------------
+local metat = { __index = {} }
+
+function open(host, port)
+    local tp = socket.try(tp.connect(host or HOST, port or PORT, TIMEOUT))
+    return setmetatable({tp = tp}, metat)
+end
+
+function metat.__index:greet()
+    return socket.try(self.tp:check(220))
+end
+
+function metat.__index:check(ok)
+    local code, status = socket.try(self.tp:check(ok))
+    return code, tonumber(socket.skip(2, string.find(status, "^%d%d%d (%d*)")))
+end
+
+function metat.__index:getdef()
+	local line = socket.try(self.tp:receive())
+	local def = {}
+	while line ~= "." do
+		table.insert(def, line)
+		line = socket.try(self.tp:receive())
 	end
-	return nil, par
+	return table.concat(def, "\n")
 end
 
-function get_def(sock)
-	local line, err = sock:receive()
-	local def = ""
-	while (not err) and line ~= "." do
-		def = def .. line .. "\n"
-		line, err = sock:receive()
+function metat.__index:define(database, word)
+    database = database or "!"
+  	socket.try(self.tp:command("DEFINE",  database .. " " .. word))
+    local code, count = self:check(150)
+	local defs = {}
+	for i = 1, count do
+  		self:check(151)
+		table.insert(defs, self:getdef())
 	end
-	if err then sock:close() return nil, err 
-	else return def end
+  	self:check(250)
+    return defs
 end
 
-function dict_open()
-	local sock, err = socket.connect("dict.org", 2628)
-	if not sock then return nil, err end
-	sock:settimeout(10)
-  	local code, par = get_status(sock, 220)
-	if code then return nil, code end
-	return sock
-end
-
-function dict_define(sock, word, dict)
-	dict = dict or "web1913"
-  	sock:send("DEFINE " .. dict .. " " .. word .. "\r\n")
-  	local code, par = get_status(sock, 150)
-	if code or not par then return nil, code end
-	local defs = ""
-	for i = 1, par do
-		local def
-  		code, par = get_status(sock, 151)
-		if code then return nil, code end
-		def, err = get_def(sock)
-		if not def then return nil, err end
-		defs = defs .. def .. "\n"
+function metat.__index:match(database, strat, word)
+    database = database or "!"
+    strat = strat or "."
+  	socket.try(self.tp:command("MATCH",  database .." ".. strat .." ".. word))
+    self:check(152)
+	local mat = {}
+	local line = socket.try(self.tp:receive())
+    while line ~= '.' do
+        database, word = socket.skip(2, string.find(line, "(%S+) (.*)"))
+        if not mat[database] then mat[database] = {} end
+        table.insert(mat[database], word)
+	    line = socket.try(self.tp:receive())
 	end
-  	code, par = get_status(sock, 250)
-	if code then return nil, code end
-	return string.gsub(defs, "%s%s$", "")
+  	self:check(250)
+    return mat
 end
 
-function dict_close(sock)
-	sock:send("QUIT\r\n")
-	local code, par = get_status(sock, 221)
-	sock:close()
-	return code
+function metat.__index:quit()
+	self.tp:command("QUIT")
+	return self:check(221)
 end
 
-function dict_get(word, dict)
-	local sock, err = dict_open()
-	if not sock then return nil, err end
-	local defs, err = dict_define(sock, word, dict)
-	dict_close(sock)
-	return defs, err
+function metat.__index:close()
+    return self.tp:close()
 end
 
-if arg and arg[1] then
-    defs, err = dict_get(arg[1], arg[2])
-    print(defs or err)
-else
-	io.write("Usage:\n  lua dict.lua <word> [<dictionary>]\n")
+-----------------------------------------------------------------------------
+-- High-level dict API
+-----------------------------------------------------------------------------
+local default = {
+    scheme = "dict",
+    host = "dict.org"
+}
+
+local function there(f)
+    if f == "" then return nil 
+    else return f end
 end
+
+local function parse(u)
+    local t = socket.try(url.parse(u, default))
+    socket.try(t.scheme == "dict", "invalid scheme '" .. t.scheme .. "'")
+    socket.try(t.path, "invalid path in url")
+    local cmd, arg = socket.skip(2, string.find(t.path, "^/(.)(.*)$"))
+    socket.try(cmd == "d" or cmd == "m", "<command> should be 'm' or 'd'")
+    socket.try(arg and arg ~= "", "need at least <word> in URL")
+    t.command, t.argument = cmd, arg
+    arg = string.gsub(arg, "^:([^:]+)", function(f) t.word = f end)
+    socket.try(t.word, "need at least <word> in URL")
+    arg = string.gsub(arg, "^:([^:]*)", function(f) t.database = there(f) end)
+    if cmd == "m" then 
+        arg = string.gsub(arg, "^:([^:]*)", function(f) t.strat = there(f) end)
+    end
+    string.gsub(arg, ":([^:]*)$", function(f) t.n = tonumber(f) end)
+    return t
+end
+
+local function tget(gett)
+    local con = open(gett.host, gett.port)
+    con:greet()
+    if gett.command == "d" then
+        local def = con:define(gett.database, gett.word)
+        con:quit()
+        con:close()
+        if gett.n then return def[gett.n] 
+        else return def end
+    elseif gett.command == "m" then
+        local mat = con:match(gett.database, gett.strat, gett.word)
+        con:quit()
+        con:close()
+        return mat
+    else return nil, "invalid command" end
+end
+
+local function sget(u)
+    local gett = parse(u)
+    return tget(gett)
+end
+
+--function socket.protect(f) return f end
+get = socket.protect(function(gett)
+    if type(gett) == "string" then return sget(gett)
+    else return tget(gett) end
+end)

etc/qp.lua

@@ -1,5 +1,11 @@
-require("ltn12")
-require("mime")
+-----------------------------------------------------------------------------
+-- Little program to convert to and from Quoted-Printable
+-- LuaSocket sample files
+-- Author: Diego Nehab
+-- RCS ID: $Id$
+-----------------------------------------------------------------------------
+local ltn12 = require("ltn12")
+local mime = require("mime")
 local convert
 arg = arg or {}
 local mode = arg and arg[1] or "-et"

makefile.dist

@@ -2,62 +2,60 @@
 # Distribution makefile
 #--------------------------------------------------------------------------
 
-DIST = luasocket-2.0-alpha
+DIST = luasocket-2.0-beta
 
 LUA = \
-	auxiliar.lua \
-	code.lua \
-	concat.lua \
 	ftp.lua \
 	http.lua \
-	select.lua \
+	ltn12.lua \
+	mime.lua \
 	smtp.lua \
+	socket.lua \
+	tp.lua \
 	url.lua
 
 TESTS = \
-	codetest.lua \
-	concattest.lua \
-	ftptest.lua \
-	mbox.lua \
-	httptest.lua \
-	noglobals.lua \
-	smtptest.lua \
 	testclnt.lua \
 	testsrvr.lua \
-	udptest.lua \
-	urltest.lua
+	testsupport.lua
 
 EXAMPLES = \
+	check-memory.lua \
+	b64.lua \
+	cddb.lua \
 	daytimeclnt.lua \
 	echoclnt.lua \
 	echosrvr.lua \
-	dict.lua \
+	eol.lua \
 	listener.lua \
+	qp.lua \
 	talker.lua \
 	tinyirc.lua
 
 ETC = \
 	check-links.lua \
-	cl-compat.lua \
+	dict.lua \
 	get.lua \
 	lua.lua \
-	luasocket.lua \
-	tftp.lua \
+	tftp.lua
 
-
-MAIN = \
+CORE = \
 	auxiliar.c \
 	auxiliar.h \
 	buffer.c \
 	buffer.h \
-	error.c \
-	error.h \
+	except.c \
+	except.h \
 	inet.c \
 	inet.h \
 	io.c \
 	io.h \
 	luasocket.c \
 	luasocket.h \
+	mime.c \
+	mime.h \
+	options.c \
+	options.h \
 	select.c \
 	select.h \
 	socket.h \
@@ -72,25 +70,56 @@ MAIN = \
 	wsocket.c \
 	wsocket.h
 
+MAKE = \
+	makefile.Darwin \
+	makefile.Linux \
+	luasocket.export \
+	mime.export \
+	luasocket.sln \
+	luasocket.vcproj \
+	mime.vcproj
+
+MANUAL = \
+	manual/dns.html \
+	manual/ftp.html \
+	manual/home.html \
+	manual/http.html \
+	manual/introduction.html \
+	manual/ltn12.html \
+	manual/luasocket.png \
+	manual/mime.html \
+	manual/reference.css \
+	manual/reference.html \
+	manual/smtp.html \
+	manual/socket.html \
+	manual/tcp.html \
+	manual/udp.html \
+	manual/url.html
+
 dist:
 	mkdir -p $(DIST)/examples
 	mkdir -p $(DIST)/tests
 	mkdir -p $(DIST)/etc
-	cp -vf $(MAIN) $(DIST)
-	cp -vf $(LUA) $(DIST)
-	cp -vf makefile $(DIST)
+	mkdir -p $(DIST)/lua
+	mkdir -p $(DIST)/make
+	mkdir -p $(DIST)/manual
+	cp -vf $(CORE) $(DIST)
 	cp -vf README $(DIST)
-	cp -vf lua.README $(DIST)
 	cp -vf NEW $(DIST)
+	cp -vf LICENSE $(DIST)
+	cp -vf $(MAKE) $(DIST)/make
+	cp -vf make.README $(DIST)/make/README
+	cp -vf $(LUA) $(DIST)/lua
+	cp -vf lua.README $(DIST)/lua/README
 	cp -vf $(EXAMPLES) $(DIST)/examples
 	cp -vf examples.README $(DIST)/examples/README
 	cp -vf $(TESTS) $(DIST)/tests
 	cp -vf tests.README $(DIST)/tests/README
 	cp -vf $(ETC) $(DIST)/etc
 	cp -vf etc.README $(DIST)/etc/README
+	cp -vf $(MANUAL) $(DIST)/manual
 	tar -zcvf $(DIST).tar.gz $(DIST)
 	zip -r $(DIST).zip $(DIST)
 
-
 clean:
 	\rm -rf $(DIST) $(DIST).tar.gz $(DIST).zip

samples/README

@@ -1,4 +1,4 @@
-This directory contains some sample  programs using LuaSocket. This code
+This directory contains some sample programs using LuaSocket. This code
 is not supported.
 
     listener.lua            -- socket to stdout
@@ -7,21 +7,26 @@ is not supported.
 listener.lua and talker.lua are about  the simplest applications you can
 write  using  LuaSocket.  Run   
 
-	'lua -l luasocket listen.lua'  and  'lua -l luasocket talk.lua'
+	'lua listen.lua'  and  'lua talk.lua'
 
-on different terminals. Whatever  you type on talk.lua will be
+on different terminals. Whatever you type on talk.lua will be
 printed by listen.lua.
 
-    dict.lua                -- dict client
+    b64.lua
+    qp.lua
+    eol.lua
 
-The dict.lua  module was  a cool  simple client  for the  DICT protocol,
-written by Luiz Henrique Figueiredo. This new version has been converted
-into a library, similar to the HTTP  and FTP libraries, that can be used
-from within  any luasocket application. Take  a look on the  source code
-and you will be able to figure out how to use it.
+These are tiny programs that perform Base64, Quoted-Printable and
+end-of-line marker conversions.
+
+    cddb.lua                -- CDDB client
+
+This is the first try on a simple CDDB client. Not really useful, but one
+day it might become a module. 
 
     daytimeclnt.lua         -- day time client
 
+
 Just run the program to retrieve the hour and date in readable form from
 any server running an UDP daytime daemon.
 
@@ -40,5 +45,9 @@ function and shows  how to create a simple server  whith LuaSocket. Just
 run tinyirc.lua and  then open as many telnet connections  as you want
 to ports 8080 and 8081.
 
+    check-memory.lua        -- checks memory consumption
+
+This is just to see how much memory each module uses. 
+
 Good luck,
 Diego.

src/auxiliar.c

@@ -16,6 +16,7 @@
 * Initializes the module
 \*-------------------------------------------------------------------------*/
 int aux_open(lua_State *L) {
+    (void) L;
     return 0;
 }
 

src/ftp.lua

@@ -176,7 +176,7 @@ end
 -- High level FTP API
 -----------------------------------------------------------------------------
 local function tput(putt)
-    local con = ftp.open(putt.host, putt.port)
+    local con = open(putt.host, putt.port)
     con:greet()
     con:login(putt.user, putt.password)
     if putt.type then con:type(putt.type) end
@@ -216,7 +216,7 @@ put = socket.protect(function(putt, body)
 end)
 
 local function tget(gett)
-    local con = ftp.open(gett.host, gett.port)
+    local con = open(gett.host, gett.port)
     con:greet()
     con:login(gett.user, gett.password)
     if gett.type then con:type(gett.type) end

src/luasocket.c

@@ -75,6 +75,7 @@ static int global_skip(lua_State *L) {
 * Unloads the library
 \*-------------------------------------------------------------------------*/
 static int global_unload(lua_State *L) {
+    (void) L;
     sock_close();
     return 0;
 }

test/README

@@ -5,19 +5,8 @@ The files provided are:
 
     testsrvr.lua            -- test server
     testclnt.lua            -- test client
-    testcmd.lua             -- test command definitions
 
-To run  the automatic  tests on  your system, make  sure to  compile the
-library  with  _DEBUG  defined  (check   makefile)  and  then  open  two
-terminals. Run  'luasocket testsrvr.lua' on  one of them  and 'luasocket
-testclnt.lua' on  the other. The  programs should start talking  to each
-other and report any failure. The tests can also be used as a benchmark.
-
-    urltest.lua             -- url.lua test module
-    codetest.lua            -- code.lua test module
-    concattest.lua          -- concat.lua test module
-
-To run these tests, just run luasocket on them and see the results.
+To run these tests, just run lua on the server and then on the client. 
 
 Good luck,
 Diego.

test/ltn12test.lua

@@ -1,5 +0,0 @@
-a = ltn12.source.file(io.open("luasocket.lua"))
-b = ltn12.source.file(io.open("auxiliar.lua"))
-c = ltn12.source.cat(a, b) 
-d = ltn12.sink.file(io.stdout)
-socket.try(ltn12.pump.all(c, d))

test/stufftest.lua

@@ -1,13 +1,13 @@
-smtp = require("smtp")
+mime = require("mime")
 
 function test_dot(original, right)
-    local result, n = smtp.dot(2, original)
+    local result, n = mime.dot(2, original)
     assert(result == right, "->" .. result .. "<-")
     print("ok")
 end
 
 function test_stuff(original, right)
-    local result, n = smtp.dot(2, original)
+    local result, n = mime.dot(2, original)
     assert(result == right, "->" .. result .. "<-")
     print("ok")
 end

test/testmesg.lua

@@ -50,10 +50,9 @@ source = smtp.message{
 
 -- finally send it
 r, e = smtp.send{
-    rcpt = "<diego@cs.princeton.edu>",
-    from = "<diego@cs.princeton.edu>",
-    source = source,
-    server = "mail.cs.princeton.edu"
+    rcpt = "<fulano@tecgraf.puc-rio.br>",
+    from = "<sicrano@tecgraf.puc-rio.br>",
+    source = source
 }
 
 print(r, e)