Iceteks Forums Migration Progress

Post by **Red Squirrel** » Mon Apr 13, 2015 10:03 pm

This is not exactly AoV related but I will keep this thread here for a general progress log of the migration process.

As I may have mentioned before I run several other forums, most are actually defunct due to too many security issues and lack of time to maintain them all, so the goal is to combine them all into a single forum, which will reside at iceteks.com. This forum included.

The forums in question are:

anythingforums.com
iceteks.com
uogateway.com
uovalor.com

All forum accounts and posts will be preserved in this transition but it will require a password reset due to the fact that all forum systems use their own method of storing passwords, it's just easier to have everyone do a reset on the new system.

Part of this migration will also involve a separate account system as the future goal is to tie in other things into it as well such as Uogateway shard submissions, and perhaps even the AoV UO accounts themselves. Also any other systems we add such as the wiki will also use the same account.

The idea behind this is to have a central account systems for all my sites and to make it easier for everyone. Also act as a central place to police spam accounts.

This thread will act as a general place for me to post progress updates.

In the meantime, please ensure your email address is up to date on your account as it will be required once the migration takes place. I have no ETA at the moment, as there is still a lot of work left to do.

Archived topic from AOV, old topic ID:6558, old post ID:38252

Post by **Red Squirrel** » Tue Apr 14, 2015 3:15 am

Sneak preview of new forum. Still in very early stages mind you. I'm most likely going to make a custom template and some forum/categories are still subject to change. I did condense a lot of them though just to keep things simpler. Topics will be merged into the most appropriate forums.

Currently the account merge process is working nicely, will start on the topic/post import tomorrow. That will be a pretty big beast as each forum software will have it's own way of handling things like bbcode, polls, attachments etc...

Oh and if you're curious how I took that screenshot, new feature I discovered in Firefox. press shift+F2 and in the bottom you'll get a command line bar, type screenshot --fullpage.

Archived topic from AOV, old topic ID:6558, old post ID:38253

Post by **Red Squirrel** » Thu Apr 30, 2015 1:29 am

Quick update... started on the topic/post migration. It's quite ugly at the moment, and I still need to do all the actual text conversion like quotes and stuff. But for now I just want to dump the text in the proper part of the DB and once I get that working I can go back and fix the actual formatting.

Here's a screenshot of doing about 50 rows:

That's just AoV posts. Currently running a full convert to see how it ends up looking. It should technically mirror this current forum as far as post order and stuff goes.

I'll then have to repeat this coding process for UOG, Iceteks and Anythingforums as all 3 use a different db format.

Oh and just dumping everything in OT for now, but there will be a translation table in the script that puts the posts in the right forums. Each site will have it's own translation table, basically.

Archived topic from AOV, old topic ID:6558, old post ID:38270

Post by **Red Squirrel** » Wed May 27, 2015 1:35 am

Been doing some good progress, I have all 4 forums merging now with basic post content. I have to code the formatting conversion stuff, and also make it put the topics in the right forums, but what I have so far is good progress.

Couple teaser screenshots:

Obviously the final merge wont just dump everything in off topic section, this is just early stages of testing.

And some very very very old posts:

Obviously, lot of formatting and other issues to fix still, but in general the content is there and is merging right.

Archived topic from AOV, old topic ID:6558, old post ID:38277

Post by **Red Squirrel** » Sat Jul 04, 2015 12:45 am

Since it's summer I have not been working much on this. Will probably resume in September when summer is over.

Archived topic from AOV, old topic ID:6558, old post ID:38297

Post by **Red Squirrel** » Thu Aug 20, 2015 1:13 am

Started working on this again. I implimented the feature in the conversion script to put the posts in all the right forums. Still a work in progress as I may add more or merge sub fourums as needed. Next step is to figure out the formatting for BB code so it converts over properly. Quick sneak preview after a conversion has been made:

And a forum view:

Nothing too special at this point, still using default template and no mods.

Once conversion is done I will work on a central authentication system as there's a few systems I want to use it such as the AoV wiki, UO Gateway shard listing, and possibly even the shard itself.

Still lot of work to do before this can be released though but conversion will be a big chunk of the work. I had to make changes to the way SMF (forum software) orders posts for the 4 way merge to work properly so I will need to test more features like post splitting to make sure they update properly.

Archived topic from AOV, old topic ID:6558, old post ID:38522

Post by **Red Squirrel** » Wed Sep 02, 2015 9:19 pm

bbcode conversion mostly complete for uovalor.com forum.

Still more testing to do with various combinations of bb code such as nested quotes/code etc. but so far so good.

I also need to convert some smilieys to other ones but that's minor.

After this I have to do IPB which two of the other forums run. Then the last one, uogateway is already running SMF so there should barely be anything to convert.

Overall doing some good progress considering I've been working on this on and off. Heading back to work tomorrow and then only get the weekend off then back to work again so wont be touching this for a while again after this.

Archived topic from AOV, old topic ID:6558, old post ID:38534

Post by **Red Squirrel** » Sun Jan 17, 2016 3:01 am

I got side tracked with other stuff, including laziness but my goal this year is to finally get this done. I also need to decide on what direction to take Iceteks, but priority is getting the new forum back up and then improving Iceteks after that.

I have some home server stuff I need to finish up, as I want to migrate some stuff off an old server box but I actually have to custom code some stuff to make it happen. Once that project is done the forums are next.

I never end up abiding by my own ETAs, but I'm going to set the forum deployment ETA to be December 2016. I'm being generous as I am hoping to do it before.

On the AoV side, this migration will also involve looking into a better account system for the shard. It will be tied into to a central authentication system, so in simple terms it means your forum account and game account will be the same. A special prefix will be added at the end of your username in UO to load the next list of characters. So say your username is dog123 and you have 3 accounts, if you want to login to the 2nd one you'd login as dog123:2 just an example though, I have not fully decided how I'll do it.

As always, ensure the email address you have set in your forum profile is valid.

Archived topic from AOV, old topic ID:6558, old post ID:38709

Post by **Red Squirrel** » Sun Aug 07, 2016 11:48 pm

About one month of summer to go, will be a really busy month for me, but after that, if things calm down, I do plan to seriously finish this. I feel bad for all the members at Iceteks and Anything forums as I had promised to have those up much sooner. I hope they'll come back once the forum is done.

Most of the work left for the migration is mostly tweaking, and then lot of testing. I also have to do the central authentication system, which while I could technically do it after, rather release it at same time.

Since this forum is the most active I will probably get some of you to test the password reset sequence at some point, I'd make a separate anouncement at that point. The merged forum + this one will be able to run at the same time for a bit, then the final migration will involve shutting down this forum and UO Gateway's forum. (all posts would be synced to new forum)

Archived topic from AOV, old topic ID:6558, old post ID:38892

Post by **Red Squirrel** » Tue Nov 15, 2016 12:28 am

I did not forget about this!

Currently Iceteks BB code conversions are being worked on.

I can't help but feel a sense of nostalgia seeing posts that are over 10 years old that's for sure. Anythingforums.com was using similar software so most of the code will be reusable.

I won't bother giving an ETA, because I keep blowing those deadlines anyway.

But I seriously need to move my butt and get this done, I actually feel bad for the old members at Iceteks.com that I basically abandoned. The downtime was never meant to be this long.

Archived topic from AOV, old topic ID:6558, old post ID:38940

Post by **Red Squirrel** » Wed Nov 23, 2016 3:25 am

I just want to take this opportunity to say that I HATE Invision Power Board's way of handling quotes. OMG what a mess. It's all hard coded HTML, instead of bbcode, so you have to parse through all that crap to get the data. preg_replace only goes so far because it errors out due to too much data exceeding the range.

That is all.

Archived topic from AOV, old topic ID:6558, old post ID:38942

Post by **Red Squirrel** » Fri Nov 25, 2016 3:52 am

Yay got quotes working! I think.... it was kinda odd as there are several variations of html, so it's quite possible I did not account for all of it.

Also got code tags working, and attachments. I can't think of anything else left to convert for Iceteks, but I could easily be overlooking stuff, so need to keep digging. Not the most efficient, but what I do is just browse posts until I see something that is off.

For attachments I kinda cheated, I just moved all the actual files to a separate "legacy" folder and then link directly to it from the post. Good enough.

Anythingforums is the last forum left to convert, and uses the same software but a different version, so the code should be mostly interchangeable I think.

I hope to do that tomorrow, (I'd do it tonight but I should really go to bed.

: )

Once I test the conversion process fully and deem it acceptable, I will start to work on the central authentication system. Rather than authenticating through the forum software it will be through a separate script. The idea is that anything I impliment, such as the Wiki on this very site will use that central authenticator instead of requiring many logins. So I need to decide how I implement it. It needs to work across servers in case I ever split stuff up. Obviously it needs to be secure too, and use an encrypted communication path if/when going across servers. So that is going to be a whole separate beast, but I don't think it will really be that hard.

Once that system is working, I will be more or less set to launch! All the stuff like modifying the template I can do after.

If I can keep up this coding rate, I am actually hoping to launch in the early new year! Maybe even before, but chances are I will reserve a few months for extensive testing, and don't want to run into or deal with issues during Christmas time.

Archived topic from AOV, old topic ID:6558, old post ID:38947

Post by **Red Squirrel** » Sun Nov 27, 2016 12:03 am

Made some decent progress in the past few days. I believe all 4 forum conversion code is working and stuff like quotes, images, etc is converting properly.

I've also been able to preserve attachments for Iceteks and Anythingforums. I cheated a bit, basically they will just be in another folder on the server and a hard link to it will end up in the post.

There is the oddball weird html stuff for Iceteks posts, but I think some of that is actually legacy stuff from previous migrations. Iceteks was originally on Yabb, then went to Invision Power Board... I think there was even another forum software before that... can't recall. It's been through many conversions.

Overall I'd say things are looking good despite the odd small glitch. Glitches that I'm not worried about. I will do more testing and polish stuff as I see fit.

Once I'm satisfied I will start on the central authentication system. I could release the forum before but I rather have it released with that system in place, so people only need to reset their passwords once.

The forum is fairly vanilla, but that is stuff I will improve on once it's released. Template, features, etc.

Archived topic from AOV, old topic ID:6558, old post ID:38948

ycontrol · Post by **ycontrol** » Wed Jan 04, 2017 1:20 pm

What's the hold up?

I am looking for some lost links and I am thinking that my signature from The Anything Forums may have one or 2...maybe, hopefully. Plus...I just miss The Anything Forums. Ahhh, those were the days...

Archived topic from AOV, old topic ID:6558, old post ID:38962

Post by **Red Squirrel** » Thu Jan 05, 2017 5:45 am

Hey long time no see! What have you been up to?

Yeah I need to move it.

The sigs won't get transferred but if you want I can send you what's in your sig as I have AF working on my local server.

Though I can probably check into transfering sigs/avatars too. Maybe even PMs. But probably not. I'll see near the end how things go.

Next step is the authentication system, it could technically wait, but it will require people to change their passwords again so rather do it in one shot.

Once I have the authentication system working then I'll finalize the forum migration script, still some quirks to work out especially on the IPB boards. Iceteks And AF. Lot of weird stuff with quote tags and such.

Now that holidays are mostly over I'll have some time to keep working on this next time I'm off.

Archived topic from AOV, old topic ID:6558, old post ID:38963

ycontrol · Post by **ycontrol** » Sun Jan 22, 2017 6:44 pm

Hey man! Good to hear from you. Don't worry about the sigs. Don't worry about anything. I don't really need that stuff.

I have been up to lots. I have a 1 year old baby girl named "Ethereal Raven Duggan". Yeah a bit of a hippie name but I was going for something that hopefully other kids didn't have...at least for a bit. Unfortunately some people like her name too much and want to steal it. We call her "Ethie" for short. She is as smart as a whip and started walking at 9 months, then moved on to climbing. Her mom is a circus trainer so she is pretty well coordinated for a 1 year old.
By the way she was born at home on the living room floor (not on purpose) but everything went great. Happy and healthy mommy and baby.

How are you?

Archived topic from AOV, old topic ID:6558, old post ID:38966

Post by **Red Squirrel** » Mon Jan 23, 2017 4:27 pm

Wow congrats! I'm doing great, I'm on my 10th year of working full time now for the phone company at the NOC (worked IT for a few years first), and had a house for around 7? years. I lost track. Time goes crazy fast these days. But pretty much same old.

Lots of projects I want to work on, so little time and motivation.

Archived topic from AOV, old topic ID:6558, old post ID:38967

ycontrol · Post by **ycontrol** » Mon Jan 23, 2017 4:57 pm

Yeah I remember you and I bought our homes right around the same time. I was following your blog when you were renovating the house and setting up servers in a section of your house that...was not previously used for computer equipment and servers....

I'm at my same job going on 13 years now, moved up to general manager. More responsibility with...more responsibility.

If you get the anything forums (or a version of it) up and going again, is there any of the old crew that will actually be around or even in the know to be active on the boards?

Archived topic from AOV, old topic ID:6558, old post ID:38968

Post by **Red Squirrel** » Mon Jan 23, 2017 8:46 pm

Yeah I'm hoping that the old crew eventually shows up. I will keep all links active so it goes to the new forum. There will be a password reset feature to get your old account back.

I slacked way too long though, I really should have done this sooner. Most people probably moved on by now, and forums in general are dying. Like even the big ones don't have as many posts as before, due to social media and stuff.

Archived topic from AOV, old topic ID:6558, old post ID:38970

Post by **Red Squirrel** » Fri Aug 11, 2017 4:53 am

Just a quick update, I am working on a completely unrelated project, but some of the code will actually be reusable for the back end of the forum authentication system. I still want to create a separate auth system for the forum so that it can be used for other stuff such as the shard itself, the uogateway shard submission system, and future stuff such as a wiki. The idea is that any system I impliment will have a single login. It will make it easier for everyone.

So while not much progress has been made on the forums as of late, I have not forgotten about this.

I sometimes wonder if I have ADD. Seriously, lol. I'm bad for getting side tracked from big projects.

In fact I'm writing this post when I should be working on something else.

Archived topic from AOV, old topic ID:6558, old post ID:39089

ycontrol · Post by **ycontrol** » Fri Oct 20, 2017 8:09 pm

In reality why would you waste time on The AF? It would probably only be me and you replying to one another once every other year...and we already do that here. So.. really.. this is The AF now.

Archived topic from AOV, old topic ID:6558, old post ID:39163

Post by **Red Squirrel** » Fri Oct 20, 2017 10:54 pm

Haha that's kinda why I have not been really putting much time into it tbh. I don't know how much traffic there really will be so kinda lack motivation. But I do want to get it done.

I do hope that some of the old folk do happen to come back once it's done but it's been years since it's been down now.

I also want to figure out a new direction for Iceteks, as it surprisingly generates some traffic even in it's dead state so having the forum back up will help.

Archived topic from AOV, old topic ID:6558, old post ID:39164

Post by **Red Squirrel** » Mon Jan 01, 2018 9:38 pm

So I'm wanting to finish this once and for all this year. Starting pretty much right away. I have a stretch of time off in a few days so will start then hopefully.

Firstly I want to look at using SSL for most of my sites including the forum, so that will be the first step, as it will give me a chance to see if I run into any issues before forum go live.

I still need to decide what direction I want to bring Iceteks but that will be a later phase. For now I just want to bring the forum back with all the posts etc.

This particular forum, and UO Gateway will also be combined into that forum. Accounts and such will remain as is, as per original plan but it will require a password reset.

Archived topic from AOV, old topic ID:6558, old post ID:39188

ycontrol · Post by **ycontrol** » Mon Jan 15, 2018 9:57 pm

Very cool. I hope all goes well. I hope some of the old folks show up and say hi...but how would they know? Wouldn't they have to be checking back now and then like you and I? If so they would end up chatting here eventually, you'd think...but there is no one else here...

Speaking of people appearing from nothing...I have a new baby coming any day now, basically. My wife's due date is Jan. 26th. So...I have a 9 year old step son (that's been with me since he was 4), a 2 year old daughter and a baby boy coming soon. Wow. Life sure sneaks up on ya

Archived topic from AOV, old topic ID:6558, old post ID:39192

Post by **Red Squirrel** » Tue Jan 16, 2018 8:54 am

I'll probably end up sending a mass email, and see how that goes.

Congrats on the upcoming baby!

Archived topic from AOV, old topic ID:6558, old post ID:39193