Hello,

I'm looking for a way to array points along a curve (in a plane) with the distance between each pair of points modulated by the pixel value of an underlying raster image (i.e. image is in same plane as curve). For example, if the image is a gradient then the spacing should smoothly decrease from the white end to the black end:



In other words, I want to do something like:

1. Fit a function on 2D plane to pixel values
2. Integrate function along curve
3. Reset integrator when integral equals a predetermined value
4. Place a point on path each time that integrator resets

while having control over the min and max distance between points such as by clipping the range of the integrand.

I'd be happy with a somewhat manual process, but can't think of how to do this with the graphics software that I know.

Hi Peer,

re: array with dependency on bitmap image luminescence, that's probably something that could be done using the Node editor:

http://moi3d.com/forum/index.php?webtag=MOI&msg=9581.122
https://moi3d.com/forum/index.php?webtag=MOI&msg=9581.57
http://moi3d.com/forum/index.php?webtag=MOI&msg=7777.1642
http://moi3d.com/forum/index.php?webtag=MOI&msg=7777.1645

Or use similar functionality in Rhino's Grasshopper node editor. It's easier to get started on this in Rhino/Grasshopper because Grasshopper has better documentation and support.

- Michael

Hi PEER,

Due to my lack of understanding, your questions seem kind of vague to me.

Are you referring to "integral-image", the Image Intensity Integral?
https://computersciencesource.wordpress.com/2010/09/03/computer-vision-the-integral-image/

Which (blocks of) pixels do you want to add up?.... in relation to your curve?

https://en.wikipedia.org/wiki/Summed-area_table

As the name suggests, the value at any point (x, y) in the summed-area table is the sum of all the pixels above and to the left of (x, y), inclusive:[3][4]
wikipedia has an algorithm.

- Brian

https://www.sciencedirect.com/topics/computer-science/intensity-value

Are the images black and white, greyscale, or color?

NOTES toward a LuminanceArrayScript

Preliminary Outline of an algorithm to array points on a curve, based upon the summation of the luminance of the underlying pixels.

Find the pseudoWidth of a pixel.
pWidth = pixelWidth; // For a horizontal curve.
// or pWidth = function of pixelWidth and pixelHeight.

Beginning with the curve startPt (left most point), Array the curve with many points, spaced by the pixelWidth.

var lumTotal = 0;

Outer For Loop to save desired final point(s) in an array.
Starting at the startPt of the curve.

Inner For loop. Add up the lumValues until lumMax is reached.
Process the next point.
get the lumValue of the underlying pixel. (Make some correction if on the boundary of pixels.)
CurrentPoint = ...
For lumTotal < lumMax;
lumTotal += lumValue;
End Inner For loop.

Add CurrentPoint to lumArray of the final desired points.

End Outer For Loop.

- Brian
I do not know the relation between the coordinates of a point on the curve, and the address of the underlying pixel.
It is probably in Wayne's script?
(This is a very rough draft.)

Brian,

Thanks for thinking about my problem!

I was referring to a path integral over a scalar field https://en.wikipedia.org/wiki/Line_integral

As I understand your outline, you're suggesting:

1. densely array points on curve
2. iterate over pixels that coincide with points on curve
3. keep only those points where lumTotal resets

There a few things I don't know how to do to make this method work:

How would I access image data of an image that is loaded into the the 3D scene? I need it to be loaded there so that I can draw the lines in the right places. In other words, I need to draw the curves manually based on the image.

Registering point coordinates and pixels, as you said. Which of Wayne's scripts do you mean?

But maybe before going into details, can you think of another way of spacing out points along a curve so the the distance between each pair of points is roughly proportional to the local luminance of an underlying image? Maybe there's a straighter path that I'm not thinking of.

Peer -

I haven't studied this, but the point spacing should be straightforward once you have the underlying image pixel data.

Take a look at Max's Heightmap script: _Heightmap.js and _Heightmap.htm

Line 30: try { moi.ui.commandUI.g_imageData = context.getImageData(0, 0, canvas.width, canvas.height).data; }

Additional Resources:

https://www.html5canvastutorials.com/advanced/html5-canvas-get-image-data-tutorial/

https://www.w3schools.com/tags/canvas_getimagedata.asp

https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/getImageData

Ed Ferguson

Yes, this is new to me also.

I was referring to ImgSampler by Wayne Hill, Jan 2020, a node. (As per Michael's links)
The nod imgSampler.js code can be examined in notepad++, and the relevant portions utilized in a regular MoI script.
That is, if we can understand what is going on. It also refers to Heightmap by Max, so maybe just look at Heightmap?

I'll study some more tonight.

- Brian

Is calculus really needed to do the luminance?

The pixel luminosity is constant in a single pixel.
So "integration" of the luminosity is accomplished by adding up the luminosity along the curve path, for each pixel. (Summation, capital Sigma Greek letter.)

Before calculus, little discrete bits were added up to get the formula for things like area, etc.

- Brian

Max's Heightmap script is not easy for me to understand.
g_imageData refers to the image pixel data.

He is getting the average of 3 colors of 1 pixel.

dX and dY are used.
?

Ed's links look promising.
I'll try to read them after work.
Nodeeditor uses a canvas, which might help.

I know little about MoI's image command and how pixels correspond to Geometry points.

- Brian

Additional resources that may help:

These JavaScript examples determine the average brightness of the entire image, but you should be able to analyze the image in segments.

https://stackoverflow.com/questions/51253752/determining-the-brightness-of-an-image-using-canvas

https://gist.github.com/codearachnid/5983831

https://codepen.io/njmcode/pen/pvLyZq

https://stackoverflow.com/questions/13762864/image-brightness-detection-in-client-side-script

Ed Ferguson

I do not see any method to get the pixel values from MoI Image, underlying some curve.
Unless there is some way to translate MoI coordinates into canvas Image coordinates.
Assume the curve is horizontal, beginning at the left of the MoI Image, and extending to the right of the MoI Image.
Let the number of horizontal pixels in the canvas image be hNum, and the number of vertical pixels in the canvas image be vNum.
Place hNum+1 points on the curve, as an array. The array index is the canvas Image x coordinate.


Figure out the vertical pixel value by dividing the image vertically into vNum+1 parts... ... to get the canvas Image y coordinate. (constant)
Figure out which vertical division the curve passes through.

Then the luminosity for each point on the curve can be looked up from the canvas Image, using the canvas methods .data.

Use the pixels values as per the algorithm outline from previous post. Fix the algorithm of course.

Max's Heightmap uses canvas context.
The pixel values are available, as per Ed's 1+ links.

- Brian

All MoI Points in some rectangle would be above one canvas image pixel.
For each canvas image pixel, there would be a rectangle in MoI, with all points in the rectangle above said pixel.

There is a lot that I do not understand.

The following link has a formula for luminosity.

https://css-tricks.com/manipulating-pixels-using-canvas/

Gray = 0.21R + 0.72G + 0.07B // Luminosity

There are several formulas for gray, and lots and lots of information...

- Brian

There is also code for a canvas full of "noise" or "static", like Max's site now shows.

Each pixel has 4 .data values.


def luminance(pixel): #returns the luminance of a pixel, used in other functions
luminance = (getRed(pixel)+getGreen(pixel)+getBlue(pixel))/3
return luminance

From 2011:
https://moi3d.com/forum/index.php?webtag=MOI&msg=4238.2
4238.2 In reply to 4238.1
Hi Janne - the background images displayed in a viewport have to be generated as a texture that is sent to your video card, and it used to be common for many video cards to have a limit on the size of textures that they supported and to also only have a limited amount of space available on them to hold the textures.

So, what is "texture", versus luminance.
A guess would be that "texture" does not have any luminance value?
Can the image "texture" below a point in MoI screen be directly acquired? Would it be of any use as "luminance" gradient?

If so, converting point coordinates from MoI, to pixel coordinates in canvas image, would be required to work from image in MoI?

Or else do all work in nodeeditor, on nodeeditor canvas.
?

Search for +ImageToRect.js does not have any results.

Found it:
https://moi3d.com/forum/index.php?webtag=MOI&msg=10730.7

- Brian

Brian,

Thanks for all of the ideas! I especially like iterating along a curve by iterating over an array of points on the curve since I had no clue about how I might do that otherwise.

I played around with HeightMap, though how it reads in image data is beyond my grasp. I haven't tried nodeeditor, but I looked at imagesampler.js and it's clearer to me there how the pixels are being read. It's good to know that I could do this on a nodeeditor canvas, though I worry that will create the problem of placing the results into the 3D world. I'll try out nodeeditor to get a feel for it.

I wrote out pseudocode over breakfast and the missing link is mapping 3D world coordinates to pixel coordinates, as you brought up in relation to texture maps. I see how I might do this with some manual input by using fiducial marks on the image, but I'm going to ask Michael if there's a way to get the coordinates of a background image directly.

Solving this problem purely in MoI may be beyond my limited abilities, so I'm going to try a last ditch solution with other software before I dig deeper into a script or node solution.

What does +imagetorect.zip do?

Ed,

Thank you for pointing out line 30 in _Heightmap.js. I'm not familiar with javascript or an html canvas, but the Mozilla link you posted gave me some inkling. I'll play around with getImageData() and see what I can do. The additional resources that you posted relating to image brightness should get me there.

Michael,

Thanks for the node links and Grasshopper recommendation. Brian and Ed have gotten me to the point of understanding that mapping between MoI 3D world units and pixels will be a problem. I could probably do it manually by putting some fiducial marks on the raster in photoshop and then manually place point objects on the fiducials as they appear in the 3D scene. I'm wondering if there's a better way.

I went through http://web.archive.org/web/20180705035034/http:/moi.maxsm.net/api and found these methods and properties related to background images:

fileName
hidden
remove
backgroundImageDrawOrder
backgroundImageShowIn
backgroundImageTransparency
getBackgroundImages
alignbackgroundimage
backgroundimage

In the GUI I can see View > Image > Properties Width and Height. Can I retrieve Width and Height in a script?

Are there any methods or properties that would allow me to find coordinates of 3 corners of a background image?

Can I find a mapping between the corners of the displayed background image and the corners of the bitmap file from which it was generated? In other words, map the corners even if the displayed background image is rotated, mirrored, or scaled.

Link to ImageToRect.js
https://moi3d.com/forum/index.php?webtag=MOI&msg=10730.7

Placed the image in a rectangle.

- Brian

function AddImage() in commands\Image.js: Link:
https://moi3d.com/forum/index.php?webtag=MOI&msg=8571.7

Hi Brian,

re:
> Search for +imagetorect.zip does not have any results.
> I downloaded imagetorect.zip on 6/24/2022.
>
> Where are the forum posts about this new script?

It's here:
http://moi3d.com/forum/index.php?webtag=MOI&msg=10730.7

- Michael

Hi Peer,

re:
> In the GUI I can see View > Image > Properties Width and Height. Can I retrieve Width and Height in a script?

Yes, they are available in script as .width and .height properties on a background image.

> Are there any methods or properties that would allow me to find coordinates of 3 corners of a background image?

Yes, the .frame property on a background image object gives a MoiCoordinateFrame object which is made up of an origin point and x/y/z axis directions forming a coordinate system. The bottom left corner of the image is located at the origin point of that frame and extends by width amount along the x axis direction and by height amount along the y axis direction.

The points can be obtained like this (warning untested code):

var lower_left = img.frame.origin;
var lower_right = img.frame.evaluate( img.width, 0 );
var upper_right = img.frame.evaluate( img.width, img.height );
var upper_left = img.frame.evaluate( 0, img.height );

- Michael