Dot Net vs VBA ... Speed and Functionality

[ettore_c]

I am looking to extract very specific information from AutoCAD and put it into Excel for manipulation, then write the updated information back to AutoCAD. At present, I am doing this with VBA, but it is HEINOUSLY slow, and it's unacceptable.

Background: My job is to manipulate and validate drawings for a litany of vendors for a whole host of things, but one consistent thing I need to do is titleblock / detail manipulation, and the "best" option I have been able to get operational is VBA inside Excel.

What my code does right now is use an array of data it needs, and where to get it (block names, the attribute names inside them, and what column of Excel to put them into). This works well enough, except almost everything I work with has ~100,000+ entities in each drawing, meaning they can take minutes each to process. The reason I am thinking .NET is considerably faster is because I have a tool that does some random stuff to these same drawings, and this tool is able to tear through many files / second; even if I made my tool do NOTHING except open and close the file, it's nowhere near as fast.

Here is some pseudocode for what my program does, for reference:
- Open file (using ObjectDBX)
-- iterate through all entites (since I can't just iterate through blocks apparently)
--- if block, see if it matches one of the blocks I want
--- if it does, but has no attributes, do something
--- if it does, but has attributes, do something else
-- next entity
- next file

Almost all of the time the program is running, it is looping through entities that are not blocks, and are therefore simply skipped. This entity search is gloriously slow, and I was curious if:
a) moving code to VB .NET is a big challenge (given my program is currently in Excel VBA, not in the AutoCAD VBA ... and I have programmed in a bunch of languages, but not VB .NET yet)
b) Simply changing my code to work in VB .NET, can expect notable speed increases (hoping in the order of a magnitude)
c) Would I be able to realise more speed increases with, say, C#
d) Does .NET offer me something like being able to iterate through just BLOCKS? Since 99.9% of my entities are just lines, this would be much quicker.
e) Does setting attributes via the .NET API have the same glitch with the shifting attributes bug of ObjectDBX? That alone almost would make this worth it.

In an ideal world, this would also become an "app", where I can open it, select the folder to process, select the desination Excel file, and go; this is basically how it is currently programmed, except Excel is the "program". However, I need it to be much faster first.

[BlackBox]

While you may not be as proficient at first, you'll find the jump to .NET quite useful in the performance department, IMHO., and well worth the investment given that you state this is something you need to do frequently.

How much performance gain? No idea, as the code needs to be written/ported to .NET, and I have no data set to even look over. Hands down, .NET API will be significantly faster than VBA, however, know that some tasks may still need to be performed using COM, depending on the limitation of .NET API features exposed.

Couple of things....

Instead of using ObjectDBX, in .NET you'd open the drawing as a side-Database using ReadDwgFile().

Unfortunately, one cannot create a SelectionSet in drawings that are not open in the editor; if you wanted to run a speed test to compare an open drawing, and another not open in the editor, consider this article for iterating nested entities.

.NET is .NET, meaning there's no speed difference between C#, VB, F#, etc. Iterating an entire database via ReadDwgFile() is pretty fast, just be sure to implement sufficient code logic so that for each ObjectId, you consider what RXClass it is, so you can skip using the Transaction to open the ObjectId (as DBObject, etc.) at all.

Without seeing your code, data set, etc. this is the best I can offer at this point.

Separately, instead of using Excel, just open VBAIDE should you choose against jumping to .NET API at this time.

HTH

[BlackBox]

You may also find this useful:

The Right Tools for the Job - AutoCAD Part 3

...

Now before you look below at the results, Stephen did mention that this was an old presentation. Looking at the old results I decided to go through and rerun all the tests so I could remember some more of the details. Here are some points you should be aware of:

1. Back then, the 64bit version of AutoCAD was not available so I had to create my own VB6 Out of Process test harness, whereas now days, simply running the same VBA code on 64bit AutoCAD gives the same results as the old VB6 exe.
2. I didn’t take care to fully optimize all of the code I wrote, however, I did tweak the Release mode ObjectARX compiler settings. I wanted to be fair to all of the languages, so I created fairly generic code that performs the same task in each language. Nevertheless, it works and I feel that it does give a good measure of relative performance between all of the languages.
3. If I remember right, all of the code was written for AutoCAD 2007. Rather than spend all my time migrating to AutoCAD 2013, I decided that the easiest path to rerunning the tests was to migrate the .NET and ObjectARX Visual Studio projects to run on AutoCAD 2011 64bit – AutoCAD 2011 64bit was the test platform on my HP Elitebook 8540w.
4. I’m not posting the code used for these tests now, because I don’t want people to focus on the code here, given that I will be discussing that in later posts.
5. Lastly, as with all computer programs, performance really depends what you are doing in your code, how you are doing it, which parts of your execution run as part of an API or actually inside of your own code and the computer you are running on. What I’m trying to say is that the actual speed of any computer program will vary – therefore you should take these results as a rough indicator, not as a definitive benchmark.

Enough details, let’s look at some data.




As I mentioned, there were three data storage categories that I ran my tests on:

1. Creating a Dictionary entry and storing an XRecord to it
2. Creating a Dictionary entry and and storing a Custom Object to it
3. Creating a Dictionary entry and storing XData to it

I created sample code for each language and ran them for 1 million cycles (performing the same operation 1 million times) with a timer routine which timed the whole process.
You’ll notice in the above slide that I have missed out the VB6/VBA (Out of Process) results – this is because they are *so* much slower than ObjectARX, .NET, VBA (In process) and LISP that if I were to apply the data to this slide you wouldn’t even see ObjectARX or .NET data showing up on the scale!
Check out why VBA developers needed to migrate their applications to VB.NET using COM Interop if they want to run on 64bit!





Of course, all of my tests showed that ObjectARX was significantly faster than all the other languages, no surprise there. That said, you can now clearly see why I don’t use Transactions in any of my sample code, it’s slower than Open/Close for both .NET and ObjectARX. I’ll dig into why this is in a later post.
What I found really surprising, is how well VBA (In Process 32bit) and LISP performed. Yes, .NET is about 5 times faster for this specific test benchmark, but considering that the time is only 75-80 seconds for 1,000,000 records, that’s pretty impressive. Looking into why this is happening, I can see that the reason is that most of the work is actually done under the hood by AutoCAD (C++ code), reinforcing my earlier point that it really depends on what you are doing whether you see dramatic (or not) resulting time differences.

[ettore_c]

Oh wow, thanks! I am sure there are optimisations to my code that I haven't seen / don't know, but if I gained 20%, it would still make my current tool "extremely slow".

I am going to try and port it over; I suppose I'll learn how stuff works once I have it operational. I use this tool almost continuously, because it is so slow (I have a second PC to run it on so it doesn't bother me with my main one ... sometimes takes DAYS ).

Thanks for the input, I'll come back when I have something working and can start looking for obvious code problems.

Cheers!

[BlackBox]

You're welcome; I'm happy to help.

Feel free to post your code if/when you get stuck, and I or someone smarter than I will be along to assist.

Cheers

[ettore_c]

Hmm, this whole "VB.NET is similar to VBA" thing is misleading; the syntax is somewhat the same, but my current program is erroneous on every single line in VB.NET. Maybe I should get a book .

EDIT: Also, though I am not 100% sure, since my code comes from Excel to process stuff in AutoCAD, I am in that "out of process" extremely tall bar shown on the chart that BlackBox kindly linked. That single graph is making me really want to get this figured out in some kind of reasonable timeframe.

[BlackBox]

I felt similarly about VB.NET, shortly after I jumped up from Visual LISP (ActiveX COM), completely skipping VBA... Since then I've learned and now pretty much code exclusively in C#, as I find it much more precise, and readable. Strange to say now, as initially I felt that way about VB.NET; interesting how tastes can change with a small amount of experience.

As for books, I only know of one for VB.NET by Jerry Winters and he hasn't kept up to date with AutoCAD AFAIK. As for C# I actually stepped away from Autodesk related products and purchased Pro C# and the .NET 4.0 Platform by Andrew Trouelsen (sp?). Very helpful to understand .NET development concepts first, and then apply those foundations to an AutoCAD environment.

Cheers

[ettore_c]

When you're working 100+ hrs / week, learning a new language is a challenge ... and the motivation to do it, since it makes my life easier, is great ... but, you know, deadlines.

I'm going to try and scoop some VB.NET examples, so I can figure out the structure of the program (this is my main challenge at this point ... simply knowing what goes where). From there, the "meat" of my program (the reading from / writing to AutoCAD) I think can be ported over easy enough ... I just don't want my new code to be all in shambles, as even my VBA code needed a re-write to make changing things easier.

I may look into C# as well, since I am re-learning anyways.

[BlackBox]

It sounds like most of this can be broken out into compartmentalized Methods... If you're able, post your code, and I'll help.

[_gile]

Hi,

The AutoCAD .NET API is much more powerfull than the COM one not only because it runs faster but because it provides more features. In your case, if you want to iterate a database looking for block references, you do not need to iterate the whole database the BlockTableRecord class provides a GetBlockReferenceIds method which allows to find all references for this block. So, you'll only need to iterate the BlockTable to get all references or only some known blocks.

About the choice of a .NET language, I do not think VB is the best one if you have a VBA background. You'll find much more (good) samples in C# and learning a completely new language should help you to see .NET as something really different from VBA, and perhaps, forget the "not so good" coding practices allowed in the VB languages family.