Math expert wanted Algorithm for finding line of two crossing areas

I wrote some code to do a similar job some years ago. Find a 'point in polygon' routine and loop through each point to find which one(s) are inside. Then from each point that is inside, work backwards until you get the point not inside. Then perform the same thing for the other polygon and you will obtain your crossing vectors. There will be lots of special cases:

two vectors with the same xy1 and xy2
two vectors one with the same xy1 and overlapping
ie (xy1 == xy1 and x2 = x2 and y2 != y2)

multiple intersections are another concern.

I suggest you presume to have a point in polygon function and then consider the variations and design logic.

I hope this helps.

Not to do exactly what you want. However, I will give you some more ideas.

Ultimately you have a collection of polygons that some may or may not overlap. You wish to merge those that overlap to create larger polygons removing the internal, redundant vectors.

What you will have to do is break down each of the polygons to a collection of vectors creating intersections where the vectors bisect.

Draw two overlapping polygons on a bit of paper. Now traverse the bounding polygon. This is in theory what you want to do. In theory it is easier than in practice. Rounding will cause errors. Not in the test environment but in real data and user intervention will be required to fix those errors.