PRI progressive revit implosion

[mschroeder]

We are swiftly moving into CD's of a residential complex that exceeds a million SF. Our largest single building is 390K SF and contains several hundred units. We sought the advice of Autodesk and others before proceeding to establish the best Revit approach to this project type and size. At that time we were advised to maintain a single Revit model; which we did. After all, we were producing a single deliverable.

This worked relatively smoothly with three and four person teams in early to mid DD. However late in DD we were pushed by a very aggressive schedule and needed to throw more bodies on the project. Soon, the team inflated to 7-8 and we started to experience what I can only explain as a progressive Revit implosion.

Back in mid DD we had seen our STC times jump to more than ten minutes. We reacted by increasing our STC interval to every 30 minutes. Slashing the STC times to about three minutes. We had also experienced out of memory crashes on several occasions, so we diligently trained the team to use selective open of worksets. Granting requests became very problematic so we adjusted to the newly enabled 8.1 behavior of transparent borrowing and instructed everyone to not "make editable" worksets. We designated a person to compact the file every night and people more or less got use to the sluggishness of the model and we sailed along.

Crunch-time, our team swelled to seven active Revit users, we encountered a troubling
sign of things to come. Our users would STC in 30 minute intervals, and almost every time they would need to wait while the central file is being accessed by someone message displayed and finally cleared. By this time STC times averaged five minutes. Suddenly, our highly productive team was left twiddling their thumbs or doing the STC dance as they call it. STC collisions were preventing transparent borrowing from working quickly and preventing our uses from saving frequently. We hit a kind of virtual wall when no matter how we timed our STC's there was always a significant wait *(see note below).

Our post mortem review of DD indicated that we had to fix this problem, or we were in serious trouble. So after many tests and scenario planning, I split the model into three separate models. Many considerations went into this split: Loss of tagging and annotation across links, duplication of types and managing type marks, dummy views and coordination of structural grids, but we needed to do it.

Now we find ourselves with cross linked models, scheduling across links and coordination management causing sleepless nights. Also troubling is we have several views that are in our base model that need to show all of these models in a single view, requiring one to load all of the Revit links. One out of two times this will crash your Revit session. Once loaded, Revit consumes nearly all of our 2GB of memory - and that's with selective workset loading.

We will forge ahead. I'm wondering if any of you are working on large Revit projects with large teams that have encountered similar issues. Any advice or support is appreciated.



* STC time: five minutes. Seven users saving every thirty minutes, (5 * 7 = 35). With seven users you would need to save every 35 minutes to save to central without colliding. As your STC interval gets longer your STC duration increases. Add the overhead of borrowing and this begins to define Revit's usability limit. What's the sweet spot for duration and interval?

Central file specs:

Single model:

180MB (flux 40MB per day with compact every night)
Split models:

Base 15MB
Building A 120MB
Building B 90MB
Structure 15MB



Additional Specs:
Revit 8.1 (latest build)
Workstation hardware: P4 3.2GHz, 2GB of RAM
Server: EMC fiber channel SAN
Network: Gigabit

[hand471037]

When's your deliverable due?

Revit 9 will help and/or solve some of your linking issues. Can't say much more, but get on the Beta if you can to see what I mean.

How many raster images do you have in there? I've found that it makes a big difference in file size.

Also, how repetitious are certain elements? Do you really need them all in the model? For example, at a place I worked doing multi-family housing, in SD thru DD we'd have all the furniture be actual 3D furniture within the units, and have all the units drawn in the building. Later on in DD and into CD's, we'd swap all the furniture in the units for very lightweight 2D families or even just lines (for it just had to be there for the building department to believe it was a livable unit, it wasn't ever seen but in plan in cd's). We'd also delete all the units on the upper floors, and instead just have one floor of 'typical' units, and tag the rest of the spaces with custom room tags that pointed back to the unit plans.

It very much sounds like you could be over-modeling. Is everything 3D? Do you have things showing up everywhere? Do you REALLY need everything to be 3D and showing up everywhere?

Also another thing is how many of your elements within your Project are referencing other things when they don't need to be? Another example from that multi-family place is where we were using the standard out of the box toilet, which is hosted by a wall. 300 units in a building equals 600 toilets equals 1200 things (the walls and the toilets) that Revit had to think about every time you touched a wall or a toilet. Not so good. It slowed things down. And it's not like we're moving the walls constantly and needing all the toilets to stay stuck to them, the toilets were within a Group anyways that was the unit plan, so it's not like someone could just move the wall, then move the toilet, and have them all change together when the finished the group. By switching to a non-wall hosted toilet it helped.

What I tell my users here is if you're not scheduling it, or counting it, and if it doesn't need to show up in every view, DON'T MODEL IT! :D

Hope this helps...

[Wes Macaulay]

I doubt Schroeder and company are over modeling: I suspect their project is just really big. We've got a couple of projects with very similar specs to yours and have exactly the same sort of problems. This is something the developers just need to address.

You either have to:

break up the model more, and/or
reduce the number of people working on any given file
...and get more RAM!

So here's my Revit-killer recipe:

Central file over 150Mb with more than 4 users working on the file
Jeffery's point about referencing is important. Keep the model as loose as possible -- lock nothing, host nothing.

[hand471037]

...and get more RAM!

Or get Autodesk to catch up with modern technology and make their tools multi-processor and 64-bit compatable. sheesh. Every year it just gets worse. Soon my new gig will be getting me a dual-core laptop, one of the new Dell ones, and Windows will barely be able to use it's new hardware, and Revit not at all. That second core will just sit idle. Or if I order that dual-opteron I'm looking at for home, and load it up with 4-6 gigs of Ram, then I'm throwing my money away for Revit won't make use of any of the extra processors or ram. But if I load Linux on it, then it's fully used. Cripes.

sorry. had to get that off my chest.

[Gordon.Price]

Or get Autodesk to catch up with modern technology and make their tools multi-processor and 64-bit compatable. sheesh. Every year it just gets worse. Soon my new gig will be getting me a dual-core laptop, one of the new Dell ones, and Windows will barely be able to use it's new hardware, and Revit not at all. That second core will just sit idle. Or if I order that dual-opteron I'm looking at for home, and load it up with 4-6 gigs of Ram, then I'm throwing my money away for Revit won't make use of any of the extra processors or ram. But if I load Linux on it, then it's fully used. Cripes.

sorry. had to get that off my chest.

Jeff,
when you get that CoreDuo (who names these things, anyway? Talk about sheesh!) try setting core two to nothing but the Revit executable. Sure Revit doesn't know anything about dual cores, but Windows does, and will try to balance threads between the cores, but I believe you can force the revit thread to one core, and let the other core handle everything else. Now RAM we are still waiting on. Hey Autodesk, just because Intel told you for years that nobody wanted 64 bit apps doesn't mean you have to keep listening. Make a 64 bit Revit; for the love of God, just do it!
I was planning on going dual core myself, but then I found a great deal on this motorcycle... ;)

Gordon

[Andre Baros]

This only helps a little bit, but we pull a lot of things out of the main file through families. A lot of our detail components are built as complete detail assemblies and then just placed into the project where they belong. The person drawing the details doesn't even need to place them, just keep drawing detail component families and let someone else place them into the model or onto sheets as needed. I gets harder with more people, but we always try to keep someone just working in the family editor. Early on on components, and later on details.

[hand471037]

Sure Revit doesn't know anything about dual cores, but Windows does, and will try to balance threads between the cores, but I believe you can force the revit thread to one core, and let the other core handle everything else.

Thanks for the tip, and trying to help out. I appreciate it. I already know this, and am gonna give it a try.

But it's just depressing that the second (Microsoft) and the fourth (Autodesk) software companies can't get their acts together to even have their software use technology that's been available for years now, let alone what's coming down the line. Heck, they (Autodesk) doesn't even seem to talk within itself at times. Just take a look at how nice the OpenGL and large models behave in Inventor as apposed to Revit. You'd think the Revit and Inventor camp would get together and swap some know-how by now... sheesh.

...and I'm not buying myself this laptop, thank god. I don't have the money either! :D

[mschroeder]

Guys:

I appreciate your feedback and I'll take it back to my camp to digest. As for looking at the next release, it's been discussed, but we think it would be more disruptive to upgrade mid CD's.

Looks like I hit a nerve with these performance limitations. Maybe one of you could start a serious methodic thread on the topic and sort the mystery from fact.




The search for the perfect venture can turn into procrastination. Your idea may or may not have merit. The key is to get started. J Bradshaw

[hand471037]

Looks like I hit a nerve with these performance limitations. Maybe one of you could start a serious methodic thread on the topic and sort the mystery from fact.

Sorry there. It's just that we've all run headlong into some of the issues and had to work around them in one way or another. And it's painful to have used Revit for five years now, to have seen it grow from zero to hero, but yet still not be able to fully utilize what now is almost commodity cheap hardware. I can get a barebones dual Opteron system for $1200 that Revit nor Windows would be able to fully use, but would be much faster and have much more ram than a ****** Dell.

Sorry to hijack the thread.

Oh hey, also, with your Project: how many linked in DWG's do you have? Are they project-wide, or just within certain views?

[mschroeder]

Oh hey, also, with your Project: how many linked in DWG's do you have? Are they project-wide, or just within certain views?

Nope, we don't allow DWG linking for performance reasons. We only import and re-import. Only a few of them are imported to a "level" the rest are peppered in drafting views.

[hand471037]

Nope, we don't allow DWG linking for performance reasons. We only import and re-import. Only a few of them are imported to a "level" the rest are peppered in drafting views.

You said that you're compacting the Central on a regular basis. Are you also Auditing and/or recreating the Central from time to time as well? Just wondering.

Also, you say that the server is on fiber, and you've got a gigabit network. What about the fileserver itself? One place I helped out with Revit found that the Revit file servers are much more heavily taxed with Revit, because the Central file actually does a ton of work when people are saving to Central on that file server itself. File servers commonly are cheap low-speed processors tied to a ton of disk. When this place moved the Revit projects to a dedicated file server that was a nice, fast server they found the save-to-central times improved. They also put that dedicated server on a switched network that was for the Revit users, and found that helped a deal as well.

[mschroeder]

You said that you're compacting the Central on a regular basis. Are you also Auditing and/or recreating the Central from time to time as well? Just wondering.


On this project it's only been nessessary to roll back a central from backup once, when we loaded a corrupt DWG that crashed the central file. I have not recreated one for maintenence purposes. Is this something you regularly? (I thought that you would loose your backup history)
Auditing always goes through without a hitch.



Also, you say that the server is on fiber, and you've got a gigabit network. What about the fileserver itself?

I'll check with our IT... it's a dual xeon 3.6.

[hand471037]

Is this something you regularly? (I thought that you would loose your backup history)

Yeah, it would loose the backup history. Making a new Central File from time to time used to be something that was a good idea back in Revit 5-6 days. I don't know if that's still the case today. Just thinking of everything I've ran into under similar situations.


I'll check with our IT... it's a dual xeon 3.6.

That should be good enough. Huh. Is it the whole office's fileserver, or just the Revit projects?

Guess you're project is just really big, and needs to get broken up a little bit more.

[mschroeder]

Is it the whole office's fileserver, or just the Revit projects?

It is a server for other than just our Revit projects, but we have several servers distibuting the project load.



Guess you're project is just really big, and needs to get broken up a little bit more.

I've only broken it up once and we are seeing acceptable performance now. I just did not want to break it up due to the interconnected nature of the beast. We are sharing many details and all family types and unit groups. These were the issues keeping it a single file.

Perhaps it is expecting to much to handle 400K SF residential building at this time. I can live with the workarounds for now. Hopefully we will start to see some of these improvements in scalability and hardware usage that you have been talking about. Until then, this is what we have.

[hand471037]

I've only broken it up once and we are seeing acceptable performance now. I just did not want to break it up due to the interconnected nature of the beast. We are sharing many details and all family types and unit groups. These were the issues keeping it a single file.

I worked for a place doing somewhat-big multi-family project 100% in Revit, so I know exactly what you're talking about here. We'd face this sort of thing with Townhouses vs. the main apartment block when doing mixed-unit style developments. I'm no longer working at that place, but some of the stuff in Revit 9 should help them (and you) with Projects of this ilk...

Would love to see some images BTW if you can share them.

[dbaldacchino]

I understand that when you have more users working on a project, you're going to have more changes that necessitate saving to central a lot more often, but I think every 35 minutes is a bit too often, especially when in CD's, you probably will have more people working on individual views rather than on the model itself. I have not run into this problem eyt (only 2 people working on a project about 173K, DD). But I might run into your same slowdown issues later on because we'll be issuing 2 sets of drawings from the same project (other project will be a similar plan with some cosmetic design changes such as windows, railings, cladding materials etc.). Why not STC every 2 hours or when significant changes occur?

[irwin]

Once loaded, Revit consumes nearly all of our 2GB of memory - and that's with selective workset loading.
If you are running into the 2GB memory limit, you should reconfigure windows xp to allow up to 3GB. Take a look at Ilya's post on this thread for more information on this: http://forums.augi.com/showthread.php?t=5563&page=3&pp=20

[mschroeder]

If you are running into the 2GB memory limit, you should reconfigure windows xp to allow up to 3GB. Take a look at Ilya's post on this thread for more information on this: http://forums.augi.com/showthread.php?t=5563&page=3&pp=20

Thanks for the tip, however, we have tested the 3GB startup switch with 4GB of RAM. There was no discernible difference, and Revit continued to crash at or around the 2GB threshold. I can't tell if it's a Windows bug or a Revit bug. All I can say is that we see no difference when running a box with 2GB RAM and one with 4GB RAM 3GB enabled.

Have you been able to run Revit in a stable manner when memory exceeds 2GB? This is a trick we are unable to pull-off.

[sbrown]

I would also like to know about this trick. I have tried it and my machine didn't work well so I turned it off. Basically I just set up my machine with the choice to boot into 3g or normal.But nothing worked in 3g.

[ilya.bass]

Thanks for the tip, however, we have tested the 3GB startup switch with 4GB of RAM. There was no discernible difference, and Revit continued to crash at or around the 2GB threshold. I can't tell if it's a Windows bug or a Revit bug. All I can say is that we see no difference when running a box with 2GB RAM and one with 4GB RAM 3GB enabled.

Have you been able to run Revit in a stable manner when memory exceeds 2GB? This is a trick we are unable to pull-off.

we have had a few customers working successfully with 3GB option. If you still experience crashes while using 3GB, please send cases to Support in order for us to investigate more closely.

Also, latest posted build of 8.1 has reduced memory consumption in some cases. if you choose to upgrade, please be sure to save all work to central, then re-save new central file and resave new local files to take full advantage of the improvement.

thanks for your patience. points about 64 bit and dual cores are valid and we are taking them into consideration. Of course I am not at liberty to disclose specific plans. multiple cores is an especially tough nut to crack - it takes very significant effort to fully utilize all cores. certain applications like games are much better suited for this, while CAD applications generally have tougher time doing it.

[hand471037]

thanks for your patience. points about 64 bit and dual cores are valid and we are taking them into consideration.

I know you can't disclose any specific plans, but I just wanted to wonder outloud about the whole fact that, as well as being not 64 bit, and not multicore, Revit is additionally tied to .Net and OpenGL. The .Net (and, well, Autodesk) pretty much means that Revit will only be available for Windows, which, well, is IMHO quickly stagnating. I've got no reason I can see yet to want to buy Vista, which was announced was just delayed again, and there's nothing there anyways that looks like it's going to help me at all. I've also heard a great deal of news that Microsoft is kind of breaking OpenGL in Vista, and pushing DirectX moreso. Great. So the primary tool I use, Revit, is tied down an OS that doesn't do jack to help me, and in the next version is going to make things harder for me while not really adding any benefits.


Of course I am not at liberty to disclose specific plans. multiple cores is an especially tough nut to crack - it takes very significant effort to fully utilize all cores.

Yeah, but you're the fourth largest software company in the world. You also make bank. You also have some very, very smart people there. Best in their field. So while I totally understand that it's a hard problem to make a bit of software multi-core, and a muchmore difficult problem to make an existing application multi-core, well, it's kinda like me saying that "it's hard to keep water out of a building, and harder to fix a building that's already leaking" when I work for one of the largest design firms around...

I don't mean any disrespect at all. It's just disheartening when the open-source and free (and volunteer-made) 3D application I use, Blender, is available on more platforms, makes better use of the hardware, and is more up to date than the very expensive BIM software I use...

[Wes Macaulay]

Hey take heart, Jeffrey -- at this rate Vista will never show up :-P

I agree with your comments, tho. And I wonder what the people at the Factory REALLY think about Windoze?

[joshg]

I know you can't disclose any specific plans, but I just wanted to wonder outloud about the whole fact that, as well as being not 64 bit, and not multicore, Revit is additionally tied to .Net and OpenGL. The .Net (and, well, Autodesk) pretty much means that Revit will only be available for Windows, which, well, is IMHO quickly stagnating. I've got no reason I can see yet to want to buy Vista, which was announced was just delayed again, and there's nothing there anyways that looks like it's going to help me at all. I've also heard a great deal of news that Microsoft is kind of breaking OpenGL in Vista, and pushing DirectX moreso. Great. So the primary tool I use, Revit, is tied down an OS that doesn't do jack to help me, and in the next version is going to make things harder for me while not really adding any benefits.

Yeah, but you're the fourth largest software company in the world. You also make bank. You also have some very, very smart people there. Best in their field. So while I totally understand that it's a hard problem to make a bit of software multi-core, and a muchmore difficult problem to make an existing application multi-core, well, it's kinda like me saying that "it's hard to keep water out of a building, and harder to fix a building that's already leaking" when I work for one of the largest design firms around...

I don't mean any disrespect at all. It's just disheartening when the open-source and free (and volunteer-made) 3D application I use, Blender, is available on more platforms, makes better use of the hardware, and is more up to date than the very expensive BIM software I use...

It may not be at all like trying to keep water out of a building, or like trying to fix a building that's already leaking. It possibly could be easier to create a new Revit program from scratch than make it support dual cores or dual processors! Even with tons of cash and brilliant developers, it is not a sure thing. Perhaps some developers out there want to say something about what is involved in upgrading a program to support dual cores or dual processors. The rest of us may not know enough about software to understand the complexities involved in this. Despite the apparent great benefits of such a feature, the resources required and time required could outweigh those benefits by a gigantic magnitude. I too, would like to see Revit support these features, although I don't think we know the details of the situation here.