I posted this message twice in the past 3 days, and it never gets on the
mailing list. Why?
Here it is again:

I have a Text field that contains paragraph text and for security reasons I
need to have it encrypted. If I do this, how can I still implement full
text search on it?
Also, I have a lot of Float columns that need to be protected but the user
has to use comparison operators like ">" and "<" on them. Any recommendations?

TIA
Mike

On 26 Oct 2007 at 9:17, mos wrote:

> I posted this message twice in the past 3 days, and it never gets on the
> mailing list. Why?
> Here it is again:
>
> I have a Text field that contains paragraph text and for security reasons I
> need to have it encrypted. If I do this, how can I still implement full
> text search on it?
> Also, I have a lot of Float columns that need to be protected but the user
> has to use comparison operators like ">" and "<" on them. Any recommendations?

Hi,

This is quite a difficult one, and as usual in the field of security depends on how valuable
the data is and how difficult you want it to be for an attacker to obtain it.

If you let us know what type of data this is and how well it has to be protected, maybe we
can more. "security reasons" is a bit vague, but I can understand that you don't want
to give too much away.

I can say one thing though, in order for the data to be indexed by MySQL , it has to be in
an unencrypted form somewhere in the database. There is no way I know to get around
this, but I hope someone can correct me

Regards

Ian
--

Ian,

At 09:36 AM 10/26/2007, you wrote:
>On 26 Oct 2007 at 9:17, mos wrote:
>
> > I posted this message twice in the past 3 days, and it never gets on the
> > mailing list. Why?
> > Here it is again:
> >
> > I have a Text field that contains paragraph text and for security
> reasons I
> > need to have it encrypted. If I do this, how can I still implement full
> > text search on it?
> > Also, I have a lot of Float columns that need to be protected but the user
> > has to use comparison operators like ">" and "<" on them. Any
> recommendations?
>
>Hi,
>
>This is quite a difficult one, and as usual in the field of security
>depends on how valuable
>the data is and how difficult you want it to be for an attacker to obtain it.
>
>If you let us know what type of data this is and how well it has to be
>protected, maybe we
>can more. "security reasons" is a bit vague, but I can understand
>that you don't want
>to give too much away.

The data is quite valuable because there is a lot of competition in this
particular marketplace and my competitors would like to get their hands on
it. I've spent 5 years writing the software and generating the data. Let's
say for the sake of argument the data is worth $1 million. How do I stop my
competitor from bribing some flunky at the ISP into turning over the backup
of my data or just e-mailing the MySQL password file to him? Also I don't
want anyone at the ISP viewing the data or changing it because I'd be
liable for any data errors.

>I can say one thing though, in order for the data to be indexed by MySQL ,
>it has to be in
>an unencrypted form somewhere in the database. There is no way I know to
>get around
>this, but I hope someone can correct me

I hope so too.
There are quite a few databases out there that have transparent encryption
(Blowfish, AES etc.) and I'm wondering why MySQL haven't implemented it,
especially now with the new laws that make the company liable for security
breaches on the web. On the other databases I've used, I haven't noticed
any speed decrease if the table is encrypted.

Mike

mos wrote:
>
> The data is quite valuable because there is a lot of competition in this
> particular marketplace and my competitors would like to get their hands
> on it. I've spent 5 years writing the software and generating the data.
> Let's say for the sake of argument the data is worth $1 million. How do
> I stop my competitor from bribing some flunky at the ISP into turning
> over the backup of my data or just e-mailing the MySQL password file to
> him? Also I don't want anyone at the ISP viewing the data or changing it
> because I'd be liable for any data errors.

Host the machines in-house. I think that could be done for less than a
million bucks for a smallish setup.

Of course, I've only ever been a bystander with that sort of project, so
the figures may be a lot higher than I'm guessing. For instance, you'd
want a beefy connection installed, of course. And then there's the
salary for someone to administer to everything.

brian

mysql@subtropolix.org wrote:
> mos wrote:
>>
>> The data is quite valuable because there is a lot of competition in
>> this particular marketplace and my competitors would like to get their
>> hands on it. I've spent 5 years writing the software and generating
>> the data. Let's say for the sake of argument the data is worth $1
>> million. How do I stop my competitor from bribing some flunky at the
>> ISP into turning over the backup of my data or just e-mailing the
>> MySQL password file to him? Also I don't want anyone at the ISP
>> viewing the data or changing it because I'd be liable for any data
>> errors.
>
> Host the machines in-house. I think that could be done for less than a
> million bucks for a smallish setup.
>
> Of course, I've only ever been a bystander with that sort of project, so
> the figures may be a lot higher than I'm guessing. For instance, you'd
> want a beefy connection installed, of course. And then there's the
> salary for someone to administer to everything.

I agree. If you're using shared hosting, forget about encryption.
Physical access to the machines ALWAYS trumps every other kind of
security, so you can't do what you're trying to do (secure data in an
insecure environment). Rent a T1 line for $500/mo and charge customers
what the data is worth.

Baron

On 10/26/07, Baron Schwartz <baron@xaprb.com> wrote:
>
> mysql@subtropolix.org wrote:
> > mos wrote:
> >>
> >> The data is quite valuable because there is a lot of competition in
> >> this particular marketplace and my competitors would like to get their
> >> hands on it. I've spent 5 years writing the software and generating
> >> the data. Let's say for the sake of argument the data is worth $1
> >> million. How do I stop my competitor from bribing some flunky at the
> >> ISP into turning over the backup of my data or just e-mailing the
> >> MySQL password file to him? Also I don't want anyone at the ISP
> >> viewing the data or changing it because I'd be liable for any data
> >> errors.
> >
> > Host the machines in-house. I think that could be done for less than a
> > million bucks for a smallish setup.
> >
> > Of course, I've only ever been a bystander with that sort of project, so
> > the figures may be a lot higher than I'm guessing. For instance, you'd
> > want a beefy connection installed, of course. And then there's the
> > salary for someone to administer to everything.
>
> I agree. If you're using shared hosting, forget about encryption.
> Physical access to the machines ALWAYS trumps every other kind of
> security, so you can't do what you're trying to do (secure data in an
> insecure environment). Rent a T1 line for $500/mo and charge customers
> what the data is worth.
>
> Baron


I also agree, however for the sake of argument could we assume that the
order of the wording in the entry probably imparts a significant amount of
it's value? If that is the case, I would think creating a second column of
unencrypted text (with a full text index) which would be nothing more than
copy of the the text with the words in a random order might provide a bit of
the protection that the user is looking for.

At 12:31 PM 10/26/2007, you wrote:
>mysql@subtropolix.org wrote:
>>mos wrote:
>>>
>>>The data is quite valuable because there is a lot of competition in this
>>>particular marketplace and my competitors would like to get their hands
>>>on it. I've spent 5 years writing the software and generating the data.
>>>Let's say for the sake of argument the data is worth $1 million. How do
>>>I stop my competitor from bribing some flunky at the ISP into turning
>>>over the backup of my data or just e-mailing the MySQL password file to
>>>him? Also I don't want anyone at the ISP viewing the data or changing it
>>>because I'd be liable for any data errors.
>>Host the machines in-house. I think that could be done for less than a
>>million bucks for a smallish setup.
>>Of course, I've only ever been a bystander with that sort of project, so
>>the figures may be a lot higher than I'm guessing. For instance, you'd
>>want a beefy connection installed, of course. And then there's the salary
>>for someone to administer to everything.
>
>I agree. If you're using shared hosting, forget about encryption.
>Physical access to the machines ALWAYS trumps every other kind of
>security, so you can't do what you're trying to do (secure data in an
>insecure environment).

Out of curiosity, why can't you use an ISP if the table is encrypted on a
dedicated server (using a database other than MySQL, say SQLite)? The pw
would be entered via VPN and is not stored in memory or in any file. The
data on disk is always encrypted and the selected rows are only decrypted
in memory on the fly. The ISP administrator never sees the pw or the
unencrypted data.

> Rent a T1 line for $500/mo and charge customers what the data is worth.

Yes, that will solve the problem. I'd have to incur more up front costs but
security would be under my control.

Mike

At 01:47 PM 10/26/2007, you wrote:
>On 10/26/07, Baron Schwartz <baron@xaprb.com> wrote:
> >
> > mysql@subtropolix.org wrote:
> > > mos wrote:
> > >>
> > >> The data is quite valuable because there is a lot of competition in
> > >> this particular marketplace and my competitors would like to get their
> > >> hands on it. I've spent 5 years writing the software and generating
> > >> the data. Let's say for the sake of argument the data is worth $1
> > >> million. How do I stop my competitor from bribing some flunky at the
> > >> ISP into turning over the backup of my data or just e-mailing the
> > >> MySQL password file to him? Also I don't want anyone at the ISP
> > >> viewing the data or changing it because I'd be liable for any data
> > >> errors.
> > >
> > > Host the machines in-house. I think that could be done for less than a
> > > million bucks for a smallish setup.
> > >
> > > Of course, I've only ever been a bystander with that sort of project, so
> > > the figures may be a lot higher than I'm guessing. For instance, you'd
> > > want a beefy connection installed, of course. And then there's the
> > > salary for someone to administer to everything.
> >
> > I agree. If you're using shared hosting, forget about encryption.
> > Physical access to the machines ALWAYS trumps every other kind of
> > security, so you can't do what you're trying to do (secure data in an
> > insecure environment). Rent a T1 line for $500/mo and charge customers
> > what the data is worth.
> >
> > Baron
>
>
>I also agree, however for the sake of argument could we assume that the
>order of the wording in the entry probably imparts a significant amount of
>it's value? If that is the case, I would think creating a second column of
>unencrypted text (with a full text index) which would be nothing more than
>copy of the the text with the words in a random order might provide a bit of
>the protection that the user is looking for.

Good point. I hadn't thought of that.

I also need to protect a couple dozen Float fields and thought I could
obscure them a bit by adding an offset to them based on an encrypted id
stored with each row. It is not going to be as good as encryption but will
to obfuscate the data.

Mike