I am reading a binary packet : 32, 8, 8, 2, 1, 1, 4, 128
I am using the following structure to parse the data:
struct header {
unsigned int a:32;
unsigned int b:8;
unsigned int c:8;
unsigned int d:2;
unsigned int e:1;
unsigned int f:1;
unsigned int g:4;
unsigned int h[4];
};
struct header head;
fread (&head, sizeof head, fp);

My data is getting garbled due to byte ordering differences. The bit
fieds are giving me incorrect values.
I suppose that I will have to use/create some byte ordering routines
( htonl / ntohl ).
I tried reading the data in a char array and then coercing it into
structure pointers so as to avoid multi-byte values but obviously the
data is dependent on how it is written and not how it is read.
Is my approach right or is their a better way to do it?

On May 28, 8:40 am, rahul <rahulsin...@gmail.com> wrote:
> I am reading a binary packet : 32, 8, 8, 2, 1, 1, 4, 128
> I am using the following structure to parse the data:
> struct header {
> unsigned int a:32;
> unsigned int b:8;
> unsigned int c:8;
> unsigned int d:2;
> unsigned int e:1;
> unsigned int f:1;
> unsigned int g:4;
> unsigned int h[4];};
>
> struct header head;
> fread (&head, sizeof head, fp);
>
> My data is getting garbled due to byte ordering differences. The bit
> fieds are giving me incorrect values.
If that struct was written to the file previously with a call to
fwrite(), it should give you the correct result.

On 28 May 2008 at 5:40, rahul wrote:
> I am reading a binary packet : 32, 8, 8, 2, 1, 1, 4, 128
> I am using the following structure to parse the data:
> struct header {
> unsigned int a:32;
> unsigned int b:8;
> unsigned int c:8;
> unsigned int d:2;
> unsigned int e:1;
> unsigned int f:1;
> unsigned int g:4;
> unsigned int h[4];
> };
> struct header head;
> fread (&head, sizeof head, fp);

fread takes 4 arguments: you need fread (&head, sizeof head, 1, fp);

You asked in another thread about packing. If you don't tell your
compiler to pack this struct then it's very likely to include one byte
of padding between g and h. Use __attribute__((packed)) in gcc.

> My data is getting garbled due to byte ordering differences. The bit
> fieds are giving me incorrect values.

It's likely that your compiler is ordering the byte containing d,e,f,g
like this:

[ . . . . . . . . ]
_______ _ _ ___
g f e d

So you may need to reorder the bit-fields in your struct.

If you also need to swap the endianness of your 32-bit fields, you can
use a macro like this:

#define CHANGE_END(x) (((x >>24) & 0x000000ff) | ((x >> 8) & 0x0000ff00) \
| ((x << 8) & 0x00ff0000) | ((x <<24) & 0xff000000) )

In article <479dcc5a-d1ef-46fe-a70b-cad902617bd2@l64g2000hse.googlegroups.com>,
<vippstar@gmail.com> wrote:

>> My data is getting garbled due to byte ordering differences. The bit
>> fieds are giving me incorrect values.

>If that struct was written to the file previously with a call to
>fwrite(), it should give you the correct result.

Only if it was written by fwrite() on the same (or a compatible)
system.

If you are transferring data between systems you need to either
mark the data with something indicating the byte order, or use a
fixed byte order. In either case you need to be able to convert the
byte order. You can do this either by manipulating the bytes manually
(with shifts and bitwise operators), or by having a set of alternative
struct definitions for the diffferent orders. Any use of structs
for this of course relies on your knowing that the system will
not mess it up by inserting padding.

-- Richard
--
In the selection of the two characters immediately succeeding the numeral 9,
consideration shall be given to their replacement by the graphics 10 and 11 to
facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)

rahul <rahulsinner@gmail.com> writes:

> I am reading a binary packet : 32, 8, 8, 2, 1, 1, 4, 128
> I am using the following structure to parse the data:
> struct header {
> unsigned int a:32;
> unsigned int b:8;
> unsigned int c:8;
> unsigned int d:2;
> unsigned int e:1;
> unsigned int f:1;
> unsigned int g:4;
> unsigned int h[4];
> };

Packet implies data communication and that usually means you want
to pay attention to writing portable code. Not 100% portable maybe
(you might be happy to assume POSIX for example) but certainly I
suggest you do this more portably than using bit fields and an int
array (you don't know the size of an int).

Often, the best way is to treat the data an array of unsigned char and
write a small set of functions that pull out data of various sizes and
representations just from the offset. These functions would use
shifts to build larger numbers from bytes and, in C, this avoids all
the endian issues.

Off-topic digression: there are often functions to transform data from
the accepted "network order" to your machine's order, but using them
means you to have, say, a long already in memory and you'll have tied
your code to you compiler's idea of a long and how it can be packed
into a struct.

--
Ben.

On Wed, 28 May 2008 09:01:46 +0000 (UTC), Antoninus Twink
<nospam@nospam.invalid> wrote:

>On 28 May 2008 at 5:40, rahul wrote:
>> I am reading a binary packet : 32, 8, 8, 2, 1, 1, 4, 128
>> I am using the following structure to parse the data:
>> struct header {
>> unsigned int a:32;
>> unsigned int b:8;
>> unsigned int c:8;
>> unsigned int d:2;
>> unsigned int e:1;
>> unsigned int f:1;
>> unsigned int g:4;
>> unsigned int h[4];
>> };
>> struct header head;
>> fread (&head, sizeof head, fp);
>
>fread takes 4 arguments: you need fread (&head, sizeof head, 1, fp);
>
>You asked in another thread about packing. If you don't tell your
>compiler to pack this struct then it's very likely to include one byte
>of padding between g and h. Use __attribute__((packed)) in gcc.
>
>> My data is getting garbled due to byte ordering differences. The bit
>> fieds are giving me incorrect values.
>
>It's likely that your compiler is ordering the byte containing d,e,f,g
>like this:
>
>[ . . . . . . . . ]
> _______ _ _ ___
> g f e d

Doing so would violate the requirement that members of the struct be
in the same order they are declared.

>
>So you may need to reorder the bit-fields in your struct.
>
>If you also need to swap the endianness of your 32-bit fields, you can
>use a macro like this:
>
>#define CHANGE_END(x) (((x >>24) & 0x000000ff) | ((x >> 8) & 0x0000ff00) \
> | ((x << 8) & 0x00ff0000) | ((x <<24) & 0xff000000) )


Remove del for email

Barry Schwarz <schwa...@dqel.com> wrote:
> Antoninus Twink wrote:
> > rahul wrote:
> > > I am reading a binary packet : 32, 8, 8, 2, 1, 1,
> > > 4, 128 I am using the following structure to parse
> > > the data:
> > > struct header {
> > > unsigned int a:32;

This violates the constraint of 6.7.2.1p2 if compiled on
an implementation where unsigned int is not at least
32-bits.

> > > unsigned int b:8;
> > > unsigned int c:8;
> > > unsigned int d:2;
> > > unsigned int e:1;
> > > unsigned int f:1;
> > > unsigned int g:4;
> > > unsigned int h[4];
> > > };
<snip>
> > > My data is getting garbled due to byte ordering
> > > differences. The bit fieds are giving me incorrect
> > > values.
> >
> > It's likely that your compiler is ordering the byte
> > containing d,e,f,g
> > like this:
> >
> >[ . . . . . . . . ]
> > _______ _ _ ___
> > g f e d
>
> Doing so would violate the requirement that members
> of the struct be in the same order they are declared.

Not necessarily. The ordering of bitfields within an
allocation unit (or even whether a bit-field overlaps
two such units) is implementation-defined [6.7.2.1p10.]

--
Peter

On May 29, 8:12 am, Peter Nilsson <ai...@acay.com.au> wrote:
> Barry Schwarz <schwa...@dqel.com> wrote:
> > Antoninus Twink wrote:
> > > rahul wrote:
> > > > I am reading a binary packet : 32, 8, 8, 2, 1, 1,
> > > > 4, 128 I am using the following structure to parse
> > > > the data:
> > > > struct header {
> > > > unsigned int a:32;
>
> This violates the constraint of 6.7.2.1p2 if compiled on
> an implementation where unsigned int is not at least
> 32-bits.
>
>
>
>
>
> > > > unsigned int b:8;
> > > > unsigned int c:8;
> > > > unsigned int d:2;
> > > > unsigned int e:1;
> > > > unsigned int f:1;
> > > > unsigned int g:4;
> > > > unsigned int h[4];
> > > > };
> <snip>
> > > > My data is getting garbled due to byte ordering
> > > > differences. The bit fieds are giving me incorrect
> > > > values.
>
> > > It's likely that your compiler is ordering the byte
> > > containing d,e,f,g
> > > like this:
>
> > >[ . . . . . . . . ]
> > > _______ _ _ ___
> > > g f e d
>
> > Doing so would violate the requirement that members
> > of the struct be in the same order they are declared.
>
> Not necessarily. The ordering of bitfields within an
> allocation unit (or even whether a bit-field overlaps
> two such units) is implementation-defined [6.7.2.1p10.]
>
> --
> Peter

I dropped the bit fields in structures and am using a bit library. The
library uses a union to determine the endian-ness
of the machine. It won't if the data is being written by some
other program but using htonl/ntohl etc serves the
purpose. The library internally uses a char to read one byte and then
returns the desired number of bits using
bitwise shift, and, or.

I had to drop the bit fields as the bits are read into a memory
address and '&' operator is not applicable for bit fields.
I initialized all the members as unsigned char. As the standard goes,
a char is always 1 byte.

rahul <rahulsinner@gmail.com> writes:
[...]
> I had to drop the bit fields as the bits are read into a memory
> address and '&' operator is not applicable for bit fields.
> I initialized all the members as unsigned char. As the standard goes,
> a char is always 1 byte.

Yes, a char is always 1 byte, but a byte is not always 8 bits.

But you're probably safe in assuming that it is on most platforms.

--
Keith Thompson (The_Other_Keith) kst-u@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

FYI
If anyone is looking for a bit library or is just curious about the
implementation, I got it from here
http://michael.dipperstein.com/bitlibs/
This library has 2 small bugs. You need to 0 the memory area before
reading where the bits are to be read
or else you will be getting incorrect results. Secondly, it does an
unnecessary right shift while reading
multiple bits.

It takes care of the endian-ness. I am posting it as I think some of
you might want to comment on the
implementation.

On 28 May, 10:01, Antoninus Twink <nos...@nospam.invalid> wrote:
> On 28 May 2008 at 5:40, rahul wrote:

> > I am reading a binary packet : 32, 8, 8, 2, 1, 1, 4, 128
> > I am using the following structure to parse the data:
> > struct header {
> > unsigned int a:32;
> > unsigned int b:8;
> > unsigned int c:8;
> > unsigned int d:2;
> > unsigned int e:1;
> > unsigned int f:1;
> > unsigned int g:4;
> > unsigned int h[4];
> > };
> > struct header head;

<snip>

> You asked in another thread about packing. If you don't tell your
> compiler to pack this struct then it's very likely to include one byte
> of padding between g and h. Use __attribute__((packed)) in gcc.

it is best to avoid non-portable constructs like this. Other
suggestions
of bit stream libraries etc. are a better way to go

<snip>


--
Nick Keighley

On Wed, 28 May 2008 20:12:33 -0700 (PDT), Peter Nilsson
<airia@acay.com.au> wrote:

>Barry Schwarz <schwa...@dqel.com> wrote:
>> Antoninus Twink wrote:
>> > rahul wrote:
>> > > I am reading a binary packet : 32, 8, 8, 2, 1, 1,
>> > > 4, 128 I am using the following structure to parse
>> > > the data:
>> > > struct header {
>> > > unsigned int a:32;
>
>This violates the constraint of 6.7.2.1p2 if compiled on
>an implementation where unsigned int is not at least
>32-bits.
>
>> > > unsigned int b:8;
>> > > unsigned int c:8;
>> > > unsigned int d:2;
>> > > unsigned int e:1;
>> > > unsigned int f:1;
>> > > unsigned int g:4;
>> > > unsigned int h[4];
>> > > };
><snip>
>> > > My data is getting garbled due to byte ordering
>> > > differences. The bit fieds are giving me incorrect
>> > > values.
>> >
>> > It's likely that your compiler is ordering the byte
>> > containing d,e,f,g
>> > like this:
>> >
>> >[ . . . . . . . . ]
>> > _______ _ _ ___
>> > g f e d
>>
>> Doing so would violate the requirement that members
>> of the struct be in the same order they are declared.
>
>Not necessarily. The ordering of bitfields within an
>allocation unit (or even whether a bit-field overlaps
>two such units) is implementation-defined [6.7.2.1p10.]

What you say is true but it seems to defeat the purpose of footnote
106: "An unnamed bit-field structure member is useful for padding to
conform to externally imposed layouts."

Yes, the footnote is for information only and your quote takes
precedence. But if the order of bit-fields within an allocation unit
is implementation defined it sure reduces the portability of any code
trying to match an externally imposed layout.


Remove del for email

In article <ou1t34lri0bubitds89t7ki9bnek6bnj6i@4ax.com>,
Barry Schwarz <schwarzb@dqel.com> wrote:

>Yes, the footnote is for information only and your quote takes
>precedence. But if the order of bit-fields within an allocation unit
>is implementation defined it sure reduces the portability of any code
>trying to match an externally imposed layout.

"The order" means high-to-low or low-to-high; it doesn't permit
shuffling them around.

-- Richard
--
In the selection of the two characters immediately succeeding the numeral 9,
consideration shall be given to their replacement by the graphics 10 and 11 to
facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)

rahul <rahulsinner@gmail.com> writes:

> FYI
> If anyone is looking for a bit library or is just curious about the
> implementation, I got it from here
> http://michael.dipperstein.com/bitlibs/
> This library has 2 small bugs. You need to 0 the memory area before
> reading where the bits are to be read
> or else you will be getting incorrect results. Secondly, it does an
> unnecessary right shift while reading
> multiple bits.
>
> It takes care of the endian-ness. I am posting it as I think some of
> you might want to comment on the
> implementation.

It does things in what is, to me, a slightly odd way. The only places
where endianness matters (for this library at least) is when reading
integer values larges than one byte. The author has chosen to do this
by exposing the endianness and using explicit multi-byte arrays. It
is much simpler (usually) to do arithmetic on the desired result type.
That method is fully portable whereas the method used by this library
works only on systems that have been catered for by the programmer (he
considers only two orders). I have written C on a machine where this
would break but of course such machines are rare nowadays, probably
because so much C code breaks on them.

--
Ben.

On 29 May 2008 12:05:21 GMT, richard@cogsci.ed.ac.uk (Richard Tobin)
wrote:

>In article <ou1t34lri0bubitds89t7ki9bnek6bnj6i@4ax.com>,
>Barry Schwarz <schwarzb@dqel.com> wrote:
>
>>Yes, the footnote is for information only and your quote takes
>>precedence. But if the order of bit-fields within an allocation unit
>>is implementation defined it sure reduces the portability of any code
>>trying to match an externally imposed layout.
>
>"The order" means high-to-low or low-to-high; it doesn't permit
>shuffling them around.

Yes, I understand that. On my system there is a status word where one
8 bit byte contains four one byte flags (f1, f2, f3, f4) and a four
byte integer (i1), in that order from left to right. If my struct
contains
int f1 : 1;
int f2 : 1;
int f3 : 1;
int f4 : 1;
int i1 : 4;
I have no guarantee that i1 follows f4. The order of the bit-fields
could be i1, f4, f3, f2, f1. And if it compiles the way I want on
version 1 of compiler 1, I have no assurance it will on version 2 or
on compiler 2. If it doesn't I cannot upgrade or switch. In order to
be strictly conforming (and therefore maximally portable) I need to
replace the bit fields with an unsigned char and use the shift and/or
and operators to extract the fields of interest. All of which seems
to defeat the purpose encouraged in footnote 106.


Remove del for email