I have rewritten my net/http script that I had questions on a couple of
days ago using httpclient but am stuck on the NTLM authentication piece.
This is what I have so far:

#!c:/ruby/bin/ruby.exe
CERT_FILE = "c:/certs/jim_nopw2.pem"
CA_CERT = "c:/ca_certs/servers_ca.cer"

require 'rubygems'
require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

client = HTTPClient.new
client.ssl_config.set_trust_ca(CA_CERT)
client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

# This fetches the page but 401 error because not authenticated through NTLM
resp = client.get("https://some_website.com/default1.asp")

# Not sure how to turn on NTLM authentication and feed arguments. I
tried this but still get 401 error
#resp = client.request('GET', "https://some_website.com/default1.asp",
{'username' => 'DOMAIN\username', 'password' => 'password'})
puts resp.content
puts resp.status

Thanks again for any and all .

-Jim

On Dec 1, 1:02 am, Jim Clark <diegosl...@gmail.com> wrote:
> I have rewritten my net/http script that I had questions on a couple of
> days ago using httpclient but am stuck on the NTLM authentication piece.
> This is what I have so far:
>
> #!c:/ruby/bin/ruby.exe
> CERT_FILE = "c:/certs/jim_nopw2.pem"
> CA_CERT = "c:/ca_certs/servers_ca.cer"
>
> require 'rubygems'
> require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed
>
> client = HTTPClient.new
> client.ssl_config.set_trust_ca(CA_CERT)
> client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)
>
> # This fetches the page but 401 error because not authenticated through NTLM
> resp = client.get("https://some_website.com/default1.asp")
>
> # Not sure how to turn on NTLM authentication and feed arguments. I
> tried this but still get 401 error
> #resp = client.request('GET', "https://some_website.com/default1.asp",
> {'username' => 'DOMAIN\username', 'password' => 'password'})
> puts resp.content
> puts resp.status
>
> Thanks again for any and all .
>
> -Jim

There are examples included in the rubyntlm gem. I'm running Windows
so they're in C:\ruby\lib\ruby\gems\1.8\gems\rubyntlm-0.1.1\examples\.
YMMV.

yermej wrote:
>> There are examples included in the rubyntlm gem. I'm running Windows
>> so they're in C:\ruby\lib\ruby\gems\1.8\gems\rubyntlm-0.1.1\examples\.
>> YMMV.
>>
I'm really trying to do this in httpclient versus getting down into
rubyntlm which shouldn't be necessary. I took another look at the
httpclient.rb source and found what I was looking for on the get method
which allows an extheader hash to be passed with arguments. Now I feel I
am real close but I am still getting a 401 error. The script is now
looking like:

#!c:/ruby/bin/ruby.exe
CERT_FILE = "c:/certs/jim_nopw2.pem"
CA_CERT = "c:/ca_certs/servers_ca.cer"

require 'rubygems'
require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

client = HTTPClient.new
client.ssl_config.set_trust_ca(CA_CERT)
client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

resp = client.get("https://some_website.com/default1.asp"), nil,
{'username' => 'DOMAIN\username', 'password' => 'password'})
puts resp.content
puts resp.status

When I look at resp.content in my debugger, I see that the @header_item
array has the normal info (content type, content length, server, etc.)
and also has ["WWW-Authenticate", "Negotiate"] and the second
["WWW-Authenticate", "NTLM"]. When I trace through the get method, I see
that @negotiate_auth has a hash @auth which == {} and @auth_default
which == null. I guess if I can figure out how to populate these
correctly then I'm probably good to go.

This will have to wait until tomorrow or Sunday night and a few more
drinks. However, I'll send another S.O.S. that if someone has already
figured this out and can show me a good code snippet, I'd be more than
willing to share a few drinks (if you are around Seattle) or offer high
praise and many thanks from afar. :-)

-Jim

Note: parts of this message were removed by the gateway to make it a legal Usenet post.

On Dec 1, 2007 4:08 AM, Jim Clark <diegoslice@gmail.com> wrote:

>
> When I look at resp.content in my debugger, I see that the @header_item
> array has the normal info (content type, content length, server, etc.)
> and also has ["WWW-Authenticate", "Negotiate"] and the second
> ["WWW-Authenticate", "NTLM"]. When I trace through the get method, I see
> that @negotiate_auth has a hash @auth which == {} and @auth_default
> which == null. I guess if I can figure out how to populate these
> correctly then I'm probably good to go.
>


Keep in mind that NTLM requires at least two round trips to the server in
each connection. Your first GET/POST/whatever request needs to have an NTLM
"type 1" message in the Authorize header. The server will then respond with
a 401, but the response will contain an NTLM "type 2" message that you use
to create an NTLM "type 3" response. You then send your GET/POST again, with
the type 3 response in the Authorize header. At that point, if all goes
well, you get a 2xx from the server.

Your best bet is to use rubyntlm as a previous commenter suggested.

On Dec 1, 12:02 am, Jim Clark <diegosl...@gmail.com> wrote:
> I have rewritten my net/http script that I had questions on a couple of
> days ago using httpclient but am stuck on the NTLM authentication piece.
> This is what I have so far:

FWIW, I've worked around this in the past by using a download of curl
that supports ntlm, and wrapped that in a shell command.

Note that if you want binary data (which it doesn't sound like you do)
and are on windows, you'll need to go through a temporary file to work
around this bug:
http://rubyforge.org/tracker/?func=d...=426&atid=1698

Francis Cianfrocca wrote:
> Keep in mind that NTLM requires at least two round trips to the server in
> each connection. Your first GET/POST/whatever request needs to have an NTLM
> "type 1" message in the Authorize header. The server will then respond with
> a 401, but the response will contain an NTLM "type 2" message that you use
> to create an NTLM "type 3" response. You then send your GET/POST again, with
> the type 3 response in the Authorize header. At that point, if all goes
> well, you get a 2xx from the server.
>
> Your best bet is to use rubyntlm as a previous commenter suggested
I admit that I'm a bit stubborn sometimes. If my wife subscribed to this
list, I'm sure she would provide ample evidence. :-)

Since the httpclient2 2.1.2 release (2007-09-22) specifically says it
supports "NTLM auth for WWW-Authenticate", I'm reluctant to give up on
it. When I look at the httpclient.rb source, the request method may loop
up to 5 times depending and the NegotiateAuth class has all the logic in
it to handle the NTLM. Tracing through, the @www_auth.set_auth method
never seems to be called at least using the client.get call. I tried to
force the set_auth call like this:

def create_request(method, uri, query, body, extheader, proxy)
if extheader.is_a?(Hash)
extheader = extheader.to_a
end
if = @_manager.find(uri)
extheader << ['', ]
end
boundary = nil
content_type = extheader.find { |key, value|
key.downcase == 'content-type'
}
if content_type && content_type[1] =~ /boundary=(.+)\z/
boundary = $1
end
req = HTTP::Message.new_request(method, uri, query, body, proxy,
boundary)
+ myuser, mypassword = nil, nil
extheader.each do |key, value|
req.header.set(key, value)
+ myuser = value if key == "username"
+ mypassword = value if key == "password"
end
+ @www_auth.set_auth(uri, myuser, mypassword) if !myuser.nil? &&
!mypassword.nil?
if content_type.nil? and !body.nil?
req.header.set('content-type', 'application/x-www-form-urlencoded')
end
req
end

And another change in line 1703 to allow access to @www_auth,
- attr_reader :www_auth
+ attt_accessor :www_auth

Sorry for not providing a real diff file. I don't have it on my windows
machine and I don't feel like fetching it at this late hour.

Point is that @www_auth.set_auth is now called within each of the
do_get_block calls on line 1956 of the request method but I am still
getting a 401.1 not authorized error.

I'm about done on following up on this though. In working through the
certificate problems for my httpclient script yesterday, I tried the
same changes in my Perl LWP script tonight and it is now working with
SSL, client certificates and NTLM authentication. Since I need a working
script, I'm moving forward now with my Perl script and dropping any more
efforts on the Ruby version. I realize I may get a few boo/hisses by
saying that on a Ruby list but I've already lost too many hours sleep
already!

If Hiroshi or anyone else gets NTLM working, please provide an example
script to show how it's done. I'd love to use this library because I
like what I see in httpclient.

Regards,
Jim

On Dec 2, 5:07 am, Jim Clark <diegosl...@gmail.com> wrote:

> If Hiroshi or anyone else gets NTLM working, please provide an example
> script to show how it's done. I'd love to use this library because I
> like what I see in httpclient.
>
> Regards,
> Jim

I don't have an NTLM server to test against at this point, but from
looking at the code you might want to try (with the original
httpclient.rb) calling set_auth on the client rather than passing in
an extra header, thusly:

CERT_FILE = "c:/certs/jim_nopw2.pem"
CA_CERT = "c:/ca_certs/servers_ca.cer"

require 'rubygems'
require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed

client = HTTPClient.new
client.ssl_config.set_trust_ca(CA_CERT)
client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)

client.set_auth(nil, 'DOMAIN\username', 'password')

resp = client.get("https://some_website.com/default1.asp")
puts resp.content
puts resp.status

I think I'll be able to play with this once I'm back at work tomorrow.
I'll post more if I can get it working. As you said, all the pieces
are there, it's just a matter of figuring out how to use them
correctly.

On Dec 2, 3:11 pm, yermej <yer...@gmail.com> wrote:
> On Dec 2, 5:07 am, Jim Clark <diegosl...@gmail.com> wrote:
>
> > If Hiroshi or anyone else getsNTLMworking, please provide an example
> > script to show how it's done. I'd love to use this library because I
> > like what I see in httpclient.
>
> > Regards,
> > Jim
>
> I don't have anNTLMserver to test against at this point, but from
> looking at the code you might want to try (with the original
> httpclient.rb) calling set_auth on the client rather than passing in
> an extra header, thusly:
>
> CERT_FILE = "c:/certs/jim_nopw2.pem"
> CA_CERT = "c:/ca_certs/servers_ca.cer"
>
> require 'rubygems'
> require 'httpclient' # using v2.1.2 and rubyntlm 0.1.1 installed
>
> client = HTTPClient.new
> client.ssl_config.set_trust_ca(CA_CERT)
> client.ssl_config.set_client_cert_file(CERT_FILE, CERT_FILE)
>
> client.set_auth(nil, 'DOMAIN\username', 'password')
>
> resp = client.get("https://some_website.com/default1.asp")
> puts resp.content
> puts resp.status
>
> I think I'll be able to play with this once I'm back at work tomorrow.
> I'll post more if I can get it working. As you said, all the pieces
> are there, it's just a matter of figuring out how to use them
> correctly.

I've played with this some and there seem to be a couple problems. Or
I'm just doing things completely wrong.

It seems that calling set_auth with the first parameter as nil should
end up setting the @auth_default instance variable in the
NegotiateAuth object. It does not. So, using the target URL as the
first parameter is the first step to getting things almost working.

Once I did that, I was getting to the response stage of the
authentication, but it wasn't successful. At this point, that seems
like it might be a problem with the way HTTPClient uses rubyntlm or it
might not support the version our server is using or...

It's a bit soon to say, but it seems that there might be a bug in
HTTPClient. I'll post again once I've figured it out.