class: center, middle
# LibCurl, The BEST web client for Perl 6 Curt Tilmes *Curt.Tilmes@nasa.gov* *The Perl Conference 2017 in DC* 2017-06-19 --- class: center, middle
# LibCurl, The BEST web client for Perl 6 ***(if you are willing to install libcurl external dependency...)*** Curt Tilmes *Curt.Tilmes@nasa.gov* *The Perl Conference 2017 in DC* 2017-06-19 --- # Introduction * **`curl`** is a popular command line tool for transferring data with URLs * **`libcurl`** is a C library encompassing the functionality: libcurl is a free and easy-to-use client-side URL transfer library, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos), file transfer resume, http proxy tunneling and more! libcurl is free, thread-safe, IPv6 compatible, feature rich, well supported, fast, thoroughly documented and is already used by many known, big and successful companies. * **`LibCurl`** is a Perl 6 module using the NativeCall capability of Perl 6 to interface directly with `libcurl` --- # LibCurl::Easy examples ``` use LibCurl::Easy; print LibCurl::Easy.new(URL => 'http://example.com').perform.content; ``` -- Let's break it up into three phases: 1 .block[.new()] Construct a new LibCurl::Easy Object You can pass in many options to control the nature of the desired transfer. The only option that is required is `URL`. You can also add options to the handle later. -- 2 .block[.perform()] Perform the transfer -- 3 .block[.content()] Examine the handle after the transfer, and, in this case, return the content. This is optional. For an upload (PUT/POST, or FTP upload), you might check status. Most methods perform some action, then return the same handle, so you can easily chain methods as in this example. --- # Setting options on an Easy handle [CAinfo](https://curl.haxx.se/libcurl/c/CURLOPT_CAINFO.html) [CApath](https://curl.haxx.se/libcurl/c/CURLOPT_CAPATH.html) [URL](https://curl.haxx.se/libcurl/c/CURLOPT_URL.html) [accepttimeout-ms](https://curl.haxx.se/libcurl/c/CURLOPT_ACCEPTTIMEOUT_MS.html) [accept-encoding](https://curl.haxx.se/libcurl/c/CURLOPT_ACCEPT_ENCODING.html) [address-scope](https://curl.haxx.se/libcurl/c/CURLOPT_ADDRESS_SCOPE.html) [append](https://curl.haxx.se/libcurl/c/CURLOPT_APPEND.html) [autoreferer](https://curl.haxx.se/libcurl/c/CURLOPT_AUTOREFERER.html) [buffersize](https://curl.haxx.se/libcurl/c/CURLOPT_BUFFERSIZE.html) [certinfo](https://curl.haxx.se/libcurl/c/CURLOPT_CERTINFO.html) [cookie](https://curl.haxx.se/libcurl/c/CURLOPT_COOKIE.html) [cookiefile](https://curl.haxx.se/libcurl/c/CURLOPT_COOKIEFILE.html) [cookiejar](https://curl.haxx.se/libcurl/c/CURLOPT_COOKIEJAR.html) [cookielist](https://curl.haxx.se/libcurl/c/CURLOPT_COOKIELIST.html) [customrequest](https://curl.haxx.se/libcurl/c/CURLOPT_CUSTOMREQUEST.html) [dirlistonly](https://curl.haxx.se/libcurl/c/CURLOPT_DIRLISTONLY.html) [failonerror](https://curl.haxx.se/libcurl/c/CURLOPT_FAILONERROR.html) [followlocation](https://curl.haxx.se/libcurl/c/CURLOPT_FOLLOWLOCATION.html) [forbid-reuse](https://curl.haxx.se/libcurl/c/CURLOPT_FORBID_REUSE.html) [fresh-connect](https://curl.haxx.se/libcurl/c/CURLOPT_FRESH_CONNECT.html) [ftp-skip-pasv-ip](https://curl.haxx.se/libcurl/c/CURLOPT_FTP_SKIP_PASV_IP.html) [ftp-use-eprt](https://curl.haxx.se/libcurl/c/CURLOPT_FTP_USE_EPRT.html) [ftp-use-epsv](https://curl.haxx.se/libcurl/c/CURLOPT_FTP_USE_EPSV.html) [ftpport](https://curl.haxx.se/libcurl/c/CURLOPT_FTPPORT.html) [header](https://curl.haxx.se/libcurl/c/CURLOPT_HEADER.html) [http-version](https://curl.haxx.se/libcurl/c/CURLOPT_HTTP_VERSION.html) [httpauth](https://curl.haxx.se/libcurl/c/CURLOPT_HTTPAUTH.html) [httpget](https://curl.haxx.se/libcurl/c/CURLOPT_HTTPGET.html) [httpproxytunnel](https://curl.haxx.se/libcurl/c/CURLOPT_HTTPPROXYTUNNEL.html) [infilesize](https://curl.haxx.se/libcurl/c/CURLOPT_INFILESIZE_LARGE.html) [low-speed-limit](https://curl.haxx.se/libcurl/c/CURLOPT_LOW_SPEED_LIMIT.html) [low-speed-time](https://curl.haxx.se/libcurl/c/CURLOPT_LOW_SPEED_TIME.html) [maxconnects](https://curl.haxx.se/libcurl/c/CURLOPT_MAXCONNECTS.html) [maxfilesize](https://curl.haxx.se/libcurl/c/CURLOPT_MAXFILESIZE_LARGE.html) [maxredirs](https://curl.haxx.se/libcurl/c/CURLOPT_MAXREDIRS.html) [max-send-speed](https://curl.haxx.se/libcurl/c/CURLOPT_MAX_SEND_SPEED_LARGE.html) [max-recv-speed](https://curl.haxx.se/libcurl/c/CURLOPT_MAX_RECV_SPEED_LARGE.html) [netrc](https://curl.haxx.se/libcurl/c/CURLOPT_NETRC.html) [nobody](https://curl.haxx.se/libcurl/c/CURLOPT_NOBODY.html) [noprogress](https://curl.haxx.se/libcurl/c/CURLOPT_NOPROGRESS.html) [nosignal](https://curl.haxx.se/libcurl/c/CURLOPT_NOSIGNAL.html) [password](https://curl.haxx.se/libcurl/c/CURLOPT_PASSWORD.html) [post](https://curl.haxx.se/libcurl/c/CURLOPT_POST.html) [postfields](https://curl.haxx.se/libcurl/c/CURLOPT_COPYPOSTFIELDS.html) [postfieldsize](https://curl.haxx.se/libcurl/c/CURLOPT_POSTFIELDSIZE_LARGE.html) [protocols](https://curl.haxx.se/libcurl/c/CURLOPT_PROTOCOLS.html) [proxy](https://curl.haxx.se/libcurl/c/CURLOPT_PROXY.html) [proxyauth](https://curl.haxx.se/libcurl/c/CURLOPT_PROXYAUTH.html) [proxyport](https://curl.haxx.se/libcurl/c/CURLOPT_PROXYPORT.html) [proxytype](https://curl.haxx.se/libcurl/c/CURLOPT_PROXYTYPE.html) [proxyuserpwd](https://curl.haxx.se/libcurl/c/CURLOPT_PROXYUSERPWD.html) [range](https://curl.haxx.se/libcurl/c/CURLOPT_RANGE.html) [redir-protocols](https://curl.haxx.se/libcurl/c/CURLOPT_REDIR_PROTOCOLS.html) [referer](https://curl.haxx.se/libcurl/c/CURLOPT_REFERER.html) [resume-from](https://curl.haxx.se/libcurl/c/CURLOPT_RESUME_FROM_LARGE.html) [ssl-verifyhost](https://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html) [ssl-verifypeer](https://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYPEER.html) [timecondition](https://curl.haxx.se/libcurl/c/CURLOPT_TIMECONDITION.html) [timeout](https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html) [timeout-ms](https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT_MS.html) [timevalue](https://curl.haxx.se/libcurl/c/CURLOPT_TIMEVALUE.html) [unrestricted-auth](https://curl.haxx.se/libcurl/c/CURLOPT_UNRESTRICTED_AUTH.html) [use-ssl](https://curl.haxx.se/libcurl/c/CURLOPT_USE_SSL.html) [useragent](https://curl.haxx.se/libcurl/c/CURLOPT_USERAGENT.html) [username](https://curl.haxx.se/libcurl/c/CURLOPT_USERNAME.html) [userpwd](https://curl.haxx.se/libcurl/c/CURLOPT_USERPWD.html) [verbose](https://curl.haxx.se/libcurl/c/CURLOPT_VERBOSE.html) [wildcardmatch](https://curl.haxx.se/libcurl/c/CURLOPT_WILDCARDMATCH.html) Throttling, proxies, cookies, multi-part forms, and much, much, more... --- # Upload/Download * .block[download => 'myfile'] * .block[upload => 'myfile'] * .block[send => 'something'] * .block[send => $mybuf] If you *don't* specify a download filename, it will stash all incoming content in `$curl.buf`, also accessible as a decoded string with `$curl.content`. --- # Info After a transfer completes (successfully or otherwise), you can access a lot of information about the transfer, statistics, headers, etc. ``` say $curl.response-code; say $curl.total-time; say $curl.speed-upload; say $curl.Content-Length if $curl.success; ``` [appconnect_time](https://curl.haxx.se/libcurl/c/CURLINFO_APPCONNECT_TIME.html) [certinfo](https://curl.haxx.se/libcurl/c/CURLINFO_CERTINFO.html) [condition-unmet](https://curl.haxx.se/libcurl/c/CURLINFO_CONDITION_UNMET.html) [connect-time](https://curl.haxx.se/libcurl/c/CURLINFO_CONNECT_TIME.html) [content-type](https://curl.haxx.se/libcurl/c/CURLINFO_CONTENT_TYPE.html) [cookielist](https://curl.haxx.se/libcurl/c/CURLINFO_COOKIELIST.html) [effective-url](https://curl.haxx.se/libcurl/c/CURLINFO_EFFECTIVE_URL.html) [ftp-entry-path](https://curl.haxx.se/libcurl/c/CURLINFO_FTP_ENTRY_PATH.html) [header-size](https://curl.haxx.se/libcurl/c/CURLINFO_HEADER_SIZE.html) [http-connectcode](https://curl.haxx.se/libcurl/c/CURLINFO_HTTP_CONNECTCODE.html) [httpauth-avail](https://curl.haxx.se/libcurl/c/CURLINFO_HTTPAUTH_AVAIL.html) [lastsocket](https://curl.haxx.se/libcurl/c/CURLINFO_LASTSOCKET.html) [local-ip](https://curl.haxx.se/libcurl/c/CURLINFO_LOCAL_IP.html) [local-port](https://curl.haxx.se/libcurl/c/CURLINFO_LOCAL_PORT.html) [namelookup-time](https://curl.haxx.se/libcurl/c/CURLINFO_NAMELOOKUP_TIME.html) [num-connects](https://curl.haxx.se/libcurl/c/CURLINFO_NUM_CONNECTS.html) [os-errno](https://curl.haxx.se/libcurl/c/CURLINFO_OS_ERRNO.html) [pretransfer-time](https://curl.haxx.se/libcurl/c/CURLINFO_PRETRANSFER_TIME.html) [primary-ip](https://curl.haxx.se/libcurl/c/CURLINFO_PRIMARY_IP.html) [primary-port](https://curl.haxx.se/libcurl/c/CURLINFO_PRIMARY_PORT.html) [proxyauth-avail](https://curl.haxx.se/libcurl/c/CURLINFO_PROXYAUTH_AVAIL.html) [redirect-url](https://curl.haxx.se/libcurl/c/CURLINFO_REDIRECT_URL.html) [request-size](https://curl.haxx.se/libcurl/c/CURLINFO_REQUEST_SIZE.html) [response-code](https://curl.haxx.se/libcurl/c/CURLINFO_RESPONSE_CODE.html) [rtsp-client-cseq](https://curl.haxx.se/libcurl/c/CURLINFO_RTSP_CLIENT_CSEQ.html) [rtsp-cseq-recv](https://curl.haxx.se/libcurl/c/CURLINFO_RTSP_CSEQ_RECV.html) [rtsp-server-cseq](https://curl.haxx.se/libcurl/c/CURLINFO_RTSP_SERVER_CSEQ.html) [rtsp-session-id](https://curl.haxx.se/libcurl/c/CURLINFO_RTSP_SESSION_ID.html) [size-download](https://curl.haxx.se/libcurl/c/CURLINFO_SIZE_DOWNLOAD.html) [size-upload](https://curl.haxx.se/libcurl/c/CURLINFO_SIZE_UPLOAD.html) [speed-download](https://curl.haxx.se/libcurl/c/CURLINFO_SPEED_DOWNLOAD.html) [speed-upload](https://curl.haxx.se/libcurl/c/CURLINFO_SPEED_UPLOAD.html) [ssl-engines](https://curl.haxx.se/libcurl/c/CURLINFO_SSL_ENGINES.html) [total-time](https://curl.haxx.se/libcurl/c/CURLINFO_TOTAL_TIME.html) --- # Multi Interface Construct an `Easy` handle for each desired transfer, then `perform` them all simultaneously. You can do hundreds of simultaneous transfers, all in a single thread. ``` use LibCurl::Easy; use LibCurl::Multi; my $curl1 = LibCurl::Easy.new(:verbose, :followlocation, URL => 'http://example.com', download => './myfile1.html'); my $curl2 = LibCurl::Easy.new(:verbose, :followlocation, URL => 'http://example.com', download => './myfile2.html'); LibCurl:Multi.new.add-handle($curl1, $curl2).perform; say $curl1.statusline; say $curl2.statusline; ``` --- # Multi Interface Async ``` use LibCurl::Easy; use LibCurl::Multi; my $curl1 = LibCurl::Easy.new(:followlocation, URL => 'http://example.com', download => 'myfile1.html'); my $curl2 = LibCurl::Easy.new(:followlocation, URL => 'http://example.com', download => 'myfile2.html'); sub callback(LibCurl::Easy $easy, Exception $e) { die $e if $e; say $easy.statusline if $easy.success; } my $multi = LibCurl::Multi.new(callback => &callback); $multi.add-handle($curl1, $curl2); $multi.perform; ``` --- # Conclusion * There are friendly short-cuts that make common actions very easy ``` my $x = get 'http://somewhere/something'; ``` * Please try it out and let me know what you like/don't like, raise issues, send pull requests, etc. * Multi interface in particular could be extended to support a more perl6-ish interface. ##.center[Thank You!]