Home
Up

Good point, I can not even suggest that I have seen any thing in writing... my opinion regarding ESRI's Open Software Concept is simply a reflection that they have made Avenue and a modest programming interface freely available to all. It is also their practice not to encrypt add-on Extensions which become available, either free or outright purchaced. Even their advanced interface (Neuron Data) is openly available to anyone who can justify the expense of such additional software. To my knowledge ESRI does not encrypt a single syllable and given enough time to research the topic, it would be very easy to create a clone of ArcView (a little too expensive to market though)?

It is this concept which attracted us to ArcView and is single handedly responsible for technical advances which we may not have be able to achieve for years. These examples act as guidelines for concepts we are developing and for that we are truly in the advantage. As our software development is not marketed (strictly inter-company) we do not concern ourselves with the encryption issue.

The reason I am in accord with you is that it is the individuals right to encryption and to know the facts regarding it's security. Publishing your finding will damage many innocents (and I personally would not want to the one who does this) but, on the other hand, it would force ESRI's hand to resolve the issue (assuming ESRI does not put a "HIT" out on you... ha). Only you could decide such a dilemma. Maybe there is a less damaging way to find out how ESRI plans to respond to such facts?

I went to your web site (which is very interesting by the way, and informative), and I didn't find the above referenced script. Please email when you load it on your web site, or please attach it to your email. I am trying to learn more about Avenue scripts.

At 09:49 PM 6/21/99 -0700, you wrote:

>it would be very easy to create a clone of ArcView (a little too expensive to market though)?

I'm surprised nobody has done it, because you're right. And it would market itself. The trick is to avoid look-and-feel protection laws. So you develop a system that has all the same capabilities as ArcView and a very readily customizable interface. Then you wait for a USER to come along and offer the Arcview interface clone. Or maybe it just appears all over the web one day. Instant marketing, instant competition.

> Publishing your finding will damage many innocents

That is the key issue. It is not so clear that any innocent people would be damaged. But if the risk is there, then, as the physicians, say, above all do no harm. That is what is causing me to hold back.

>Maybe there is a less damaging way to find out how ESRI plans to respond to such facts?

I'm working on it. However, in my experience, ESRI rarely responds to facts and readily responds to marketing opportunities. This is not a marketing opportunity. Even though my message went through the ESRI list server and through a human moderator it has not (yet) generated an ESRI response. I have tried other channels, to no avail. I am trying to go through a third party.

You go, boy! Have you noticed that ESRI is getting a little too Microsoft-ish? I emailed them asking for advice on how not to let my projects get corrupted by the dreaded "AV Array: Index 1 not in range 0..0" and they said I had used up my "courtesy" support call and that it would cost me. Imagine! I'll have to pay to help them resolve their bugs! At least I've found a method of fixing dead projects but I still can't prevent it ... Sure is embarrassing when I go to pull up a project to show the boss and it won't open!

As a developer you must be aware that decompilers exist for almost everything and that is possible to reverse engineer any software out there given sufficient time and effort. That is exactly why most software licenses include language which specifies it is illegal to reverse engineer or decompile the supplied software product.

If you are in deed a developer and place such a product out there in the public domain, I am quite sure you will eventually find your self and company in court one day. I am also not sure what could possibly be your motivation for placing such a piece of software out in the public forum. Your only choice would be to not develop ESRI products.

Also I recommend that you be careful of your actions. ESRI does not have to continue to license their product to you or your company. I am sure that the action you describe here would result in numerous developers asking for your removal. While ESRI might need to improve its current product, destroying the value of existing software developed in Avenue would not be good for the industry.

I'm aware that probably many of us want your script to decrypt encrypted Avenue-scripts. However, I would like to ask if you can send me one, or otherwise if you can send me a description of how to do it. I couldn't find it at the ESRI-site nor the alternative site you mentioned in your mail.

I read your contribution of the Arcview discussion group about the possibility to unencrypt project with a specific script in Avenue. Did you put this script already on the ESRI script page or will it be available on your Web page?

At 03:01 PM 6/22/92 +0200, you wrote:

>I did not see your code for "unencrypting" encrypted scripts neither on ESRI site nor on yours (http://members.home.net/whuber), but I believe you have it (if you say so). Now, do you have any idea how one can protect the code of his scripts.

A lot of people have written asking (that's the polite word) me not to post this code. I am considering their requests, so no code has been posted. I will update you shortly on the responses I have received and what action I will take. Nevertheless, whether I publish this or not, you may as well assume anyone can read your encrypted scripts.

One person wrote back with this suggestion:

"If you really want to buffalo someone from stealing avenue code, write it so that it simulates mouseclicks on dialogs that aren't shown. Jeez, debugging that will drive you mad."

That's a very interesting and clever idea, but probably impractical. Another suggestion is to compile the key parts into a DLL using another language, where you get as much protection as anybody has. (There exist decompilers, object debuggers, and so on that effectively "decrypt" DLLs and EXEs.) A lot of Avenue code is not amenable to this approach, though: it's good only for certain kinds of algorithms, but not for how you build an interface, manage the data, and so on.

> I believe you have it (if you say so).

I am glad you are sceptical (not enough people are). Feel free to send me a project or extension file with encrypted scripts. If you can prove those scripts are yours--for example, if your name and e-mail address are embedded as comments within them--I will e-mail you the decrypted versions.

I agree with most of your points (although the implicit threats do not add to their persuasiveness). I am curious, though, how you conclude that publishing what could be construed as a "decompiler" would

"[destroy] the value of existing software developed in Avenue".

That's an amazing assumption, given that (as many respondents have pointed out to me) the vast majority of software developed in Avenue is not encrypted, including every ESRI extension. It's a key assumption: if you (or anyone else) can demonstrate that clarifying ArcView's encryption shortcomings would actually destroy software value, I will certainly go no further. Note, please, that pending a resolution of issues such as those you bring up, I have not posted anything to the web.

At 03:15 PM 6/22/99 +0200, you wrote:

>However, I would like to ask if you can send me one, or otherwise if you can send me a description of how to do it. I couldn't find it at the ESRI-site nor the alternative site you mentioned in your mail.

I would like nothing more than to be able to satisfy your request right now. Several people, however, have sent me urgent messages alleging that great harm would result if I did so. I am waiting for clarification from them before deciding how to proceed, since I am bound by a professional code of ethics that asserts "avoid harm to others" (http://csep.iit.edu/codes/coe/ACM-CoE.htm).

I will get back to you, probably through ArcView-L, within a few days.

At 11:06 AM 6/22/99 -0700, you wrote: >i'm an arcview user that would very much like to unencrypt a script that i have in order to alter and expand on how it works

I would like nothing more than to help you this instant. However, several ArcView developers have written to me asserting that offering decrypting capability would seriously harm or damage them. This raises ethical and even legal issues (one developer was overtly threatening) that have to be resolved before I can go further. The ideal solution would be one that gives you the decryption capability you want without hurting anyone else. Any suggestions?

At 11:43 AM 6/22/99 -0700, you wrote:

>The script I wish to look at is actually one that is offered for use through the esri web site

Which script might that be? Perhaps there's a way to satisfy your request; I would need to look at the web site first. Do you have the reference?

From: Respondent 17 To: Bill Huber Date: 22 Jun 1999 17:00:00 +0100

I was interested to read that one can decrypt, encrypted scripts. I have tried to access the site you mention on web page.http://members.home.net/whuber. but it throws up an error and asks one to inform the owner. May I ask, is it possible for you to email me a copy of the script.

At 05:00 PM 6/22/99 +0100, you wrote: >May I ask, is it possible for you to email me a copy of the script.

I would like nothing more than to help you this instant. However, several ArcView developers have written to me asserting that offering decrypting capability would seriously harm or damage them. This raises ethical and even legal issues (one developer was overtly threatening) that have to be resolved before I can go further. Thus, there is presently nothing on my web site related to Avenue decryption. The ideal solution would be one that gives you the decryption capability you want without hurting anyone else. I am looking for good suggestions and would love to hear your further thoughts on the matter.

I was trying to find the script ref reading encrypted scripts.

I lost the original of one of my scripts months ago and have been unable to update it

Could you send me the URL of the site please?

Last weekend I stumbled upon the weakness of the Avenue encryption scheme simply by glancing at several encrypted scripts. They shared the characteristic features you describe in your paper ...; the encryption method--and hence the decryption method--were then immediately apparent. ...  I quickly posted a claim to ArcView-L that the encryption is weak and asserted that developers should not rely on it. To push the point, I offered to post working decryption code on the web. (That was probably a mistake, but I don't want to get sidetracked here. Suffice it to say I have not done so.) This provoked many responses, some heated. Only one respondent was aware of the information in your paper. It should be better known!

So here's my question. You discuss the VB scheme (essentially a p-code interpreter) and attribute its insecurity "to the ignorance and incompetence of the programming team." In the next section you discuss the Avenue and AML schemes, but conclude "the developers at ESRI are not to blame for these weak encryption schemes..." I think it's fair to say that decompiling p-code requires quite a bit more effort than deciphering encrypted Avenue or AML, so why do you come to such radically different conclusions? Should you not conclude the ESRI developers are even more incompetent than the VB programming team?

With you, I completely agree that "fault can be found in promoting such a weak algorithm as 'irreversible'; this is dishonest, and does little except create further confusion." Yet I find it hard to believe that ESRI management, technical writing, and sales people would knowingly make dishonest claims. It is in ESRI's best interests to support their third party developers, which includes providing reasonably strong encryption for scripts. Furthermore, there is no technical barrier to greatly enhancing the security of the encryption within the limitations of all export restrictions. The simple expedient of XORing a deterministic stream of pseudo-random bits (established with a random seed every time) with the plaintext (or better yet, with a lexically analyzed version of the original script to remove comments, etc.) would produce something relatively easy for a diligent expert to penetrate but quite hard for anyone else to decipher. Encryption and decryption would be just as fast to execute and easy to program as the current method. I conclude that the ESRI development team (or, more likely, person) was simply incompetent or lazy or both.

I am getting sidetracked. Finding fault is not to the point (although knowing the cause of a software failure, which this weak encryption truly is, can improve future development). I am curious, though, whether you shared your paper or its conclusions with ESRI and if so, when. I have had little success providing any similar information to ESRI and want to avoid repeating whatever you have tried.

At 05:23 PM 6/22/99 +0100, you wrote: >I was trying to find the script ref reading encrypted scripts. I lost the original of one of my scripts months ago and have been unable to update it Could you send me the URL of the site please?

I would like nothing more than to help you this instant. Your problem is exactly the one my extension solves. However, several ArcView developers have written to me asserting that offering decrypting capability would seriously harm or damage them. This raises ethical and even legal issues (one developer was overtly threatening) that have to be resolved before I can go further. The ideal solution would be one that gives you the decryption capability you want without hurting anyone else. Any suggestions?

In the meantime, based on a hint dropped my one respondent, I found a reasonably accurate description of the encryption (and decryption) methods already on the web. ... Since you can write scripts, you should have little trouble converting that description into code to recover yours.

I have freely offered to decrypt any script anybody sends me, provided they can demonstrate ownership (such as through an embedded name or e-mail address in the comments). If your script is like that, just send me the project (or avx or odb) file and I'll be happy to send you back a project containing all the scripts--decrypted--contained in that file.

i'm an arcview user that would very much like to unencrypt a script that i have in order to alter and expand on how it works... can you help me out? i have looked on your scratch web site page for the unencrypt script and on the esri arcscript page but cannot find it...

At 01:38 PM 6/22/99 -0700, you wrote:

' Philip N. Hooge and Bill Eichenlaub
' USGS-BRD Glacier Bay Field Station
' Glacier Bay Field Station
' Gustavus, AK
' philip_hooge@usgs.gov
' http://www.absc.usgs.gov/glba/phooge.htm

(that's copied right out of View.SpiderDiagram).

Some are simple modifications of ESRI scripts (View.PointToPolyline). If I have understood the comments to View.MCP correctly, I could probably provide you original fast clean code for that. But in general, why don't you contact Hooge at the e-mail address in the comments and ask whether it would be all right to view his source for educational or research purposes. He would probably be delighted.

I understand the position you're in, and I am not surprised that many have had the reaction you described... I did not intend to put you in an awkward position...

The script I wish to look at is actually one that is offered for use through the esri web site so I see no reason why it was encripted to begin with and no harm to the author in modifying it to work efficiently within another's project... infact, I have already acknowledged him as the source... I just saw it as a shortcut to reinventing the wheel, so to speak...

On Tue, 22 Jun 1999, Bill Huber wrote:

> Therefore I quickly posted a claim to ArcView-L that  the encryption is weak and asserted that developers should not rely on it.  To push the point, I offered to post working decryption code on the web.  (That was probably a mistake, but I don't want to get sidetracked here.  Suffice it to say I have not done so.) This provoked many responses, some  heated. Only one respondent was aware of the information in your paper.  It should be better known!

heres my story - I discovered this in a similar way to you, going through some apr files with a text editor to change pathnames. I did post it on the esri listserves and comp.infosystems.gis some three years ago. I also wrote a little -program ... which did decompile the apr and offered it to anyone who was interested. I got about 50 replies including some from esri. I had nothing to hide so I sent it to them too.

I got a call from the projct manager for arcview a few days later - I think his name is rich turner. He asked that I stop distributing the program, suggesting it was illegal. I agreed to do this, since I felt the point had been made. I had friends working internally to ESRI who described the scene as a madhouse - the developer who wrote the code took it especially hard, and wrote a mean spirited piece on my intentions for the Harlow Report, calling me the 'Unacracker' (this was right at the time of the Unabombers initial discovery)

> > So here's my question. You discuss the VB scheme (essentially a p-code   interpreter) and attribute its insecurity "to the ignorance and   incompetence of the programming team." In the next section you discuss the Avenue and AML schemes, but conclude "the developers at ESRI are not to blame for these weak encryption schemes..." I think it's fair to say that decompiling p-code requires quite a bit more effort than deciphering  encrypted Avenue or AML, so why do you come to such radically different conclusions? Should you not conclude the ESRI developers are even more incompetent than the VB programming team?

This article was written after the above incident, so I toned down the esri part a bit. ESRIs policy is definitely to cover it up. The problem was first reported under Arcview 2.1, but they still have not fixed it in 3.0,3.0a,3.0b, or 3.1. Since AV is not backward compatible I wonder why is still has not been fixed - it may be very difficult.

> > With you, I completely agree that "fault can be found in promoting such a  weak algorithm as 'irreversible'; this is dishonest, and does little except   create further confusion." Yet I find it hard to believe that ESRI   management, technical writing, and sales people would knowingly make   dishonest claims.

This is definitely what the policy is.

> >It is in ESRI's best interests to support their third  party developers, which includes providing reasonably strong encryption for  scripts. Furthermore, there is no technical barrier to greatly enhancing  the security of the encryption within the limitations of all export  restrictions.

Yes - this has also changed recently with the export standards.

> >The simple expedient of XORing a deterministic stream of pseudo-random bits (established with a random seed every time) with the plaintext (or better yet, with a lexically analyzed version of the original script to remove comments, etc.) would produce something relatively easy for a diligent expert to penetrate but quite hard for anyone else to decipher. Encryption and decryption would be just as fast to execute and easy to program as the current method. I conclude that the ESRI development team (or, more likely, person) was simply incompetent or lazy or both.

With the advent of the Internet, I think encryption is very difficult to implement - all of the schemes for all of the major products have been broken and many vendors are selling the decryption tools. The problem is if just one person spends the time to break it, the world can then know. I dont think avenue is high-powered enough to contain any serious secrets either.

> > I am getting sidetracked. Finding fault is not to the point (although knowing the cause of a software failure, which this weak encryption truly is, can improve future development). I am curious, though, whether you shared your paper or its conclusions with ESRI and if so, when. I have had little success providing any similar information to ESRI and want to avoid repeating whatever you have tried.

They are aware of the problem for three years now. About 1-2 months ago, I received a letter similar to yours from a professor in England who had also solved the problem on his own. He was considering selling a decryption prouct. ESRI-England preferred that he not do this, I dont know what he did.

It is interesting to note that most of that article has become true - there is now massive piracy of GIS applications and datasets on the web, including the infamous CD-burning factory in Russia which is mentioned on ESRIs home page. Most AVENUE developers are releasing fully-functional demos on the web relying upon the AVENUE encryption. I did write to some of them btu they took no heed - perhaps they figure that distribution ultimately will lead to more sales.

I have received some requests for the program which I dont distribute anymore. Some of these have been clearly legitmate where the client had bought a customized gui from a developers who was now out of business, and the gui didnt work, had bugs, etc. Anyways, I have had my go-around with this, if you could try and keep references to me as few as possible, I would appreciate it.

Note: this slightly edited extract of the exchange with Respondent 20 has been posted with permission.

the site you might be looking for is:

the script is part of the animal movement extension written by: Philip N. Hooge Research Population Ecologist USGS-Alaska Biological Science Center Glacier Bay Field Station Gustavus, AK 99826

the details of his development are at:

and the scripts you may want to look at are: View.MCP View.PointToPolyline View.SpiderDiagram View.Jennrich_Turner

Respondents 1-10    Respondents 20-27   Respondents 28-33     ArcView Encryption

ColorRamp, Memorized Calculations, Rotate, Sample, XSect, and Tissot  are  trademarks of Quantitative Decisions.  All other products mentioned are registered trademarks or trademarks of their respective companies.

Questions or problems regarding this web site should be directed to webmaster@quantdec.com.
Copyright © 2000-2002 Quantitative Decisions.  All rights reserved.
Last modified: Monday July 01, 2002.