affair

That really depends on what state you live in.  In some states, the adultery you committed could cost you more than just your half of the home, if he files for divorce.  You could be determined "at fault", and lose quite a bit.  Spaznskitz is a family law attorney, and can probably give you some advice, if you say what state you're in.  Quite a mess, but not beyond the point of no return.  Do you want to work things out with your husband, are you willing to grovel and do whatever he needs you to do?  Is the other relationship over?

The laws differ by state.  Depending on how long you have been out of the home it may be abandonment or you may still have rights to your share of the property, monetarily.  But because you are not living in the home I think it's likely that legally you would not be allowed to move back in.  Don't forget that he also has rights to the children who obviously want to see him.  You have not said whether or not you intend to stay separated and just want the property you are entitled to, or if you are hoping to reconcile.  Reconciliation is very hard; I am in the middle of it myself.  Don't do it just for the kids.  You have to want it yourself, and your ex has to want it just as badly.  It sounds as if he has closed himself off to this possibility.  You should definitely consult an attorney, especially because there are children involved.

Think you gave them up.  Doe's he want you back? Or where you just planning on forcing yourself back in?
Will he go to counseling? I s he willing to forgive you? You need to talk to him, beg and plead if you want back. You made a big mistake and people make mistakes. Personally that  a mistake I wont accept  but I don't know you or your husband.